| Gcs Debug state: | |
| GcsNodeManager: | |
| - RegisterNode request count: 1 | |
| - DrainNode request count: 0 | |
| - GetAllNodeInfo request count: 38 | |
| GcsActorManager: | |
| - RegisterActor request count: 0 | |
| - CreateActor request count: 0 | |
| - GetActorInfo request count: 0 | |
| - GetNamedActorInfo request count: 0 | |
| - GetAllActorInfo request count: 0 | |
| - KillActor request count: 0 | |
| - ListNamedActors request count: 0 | |
| - Registered actors count: 0 | |
| - Destroyed actors count: 0 | |
| - Named actors count: 0 | |
| - Unresolved actors count: 0 | |
| - Pending actors count: 0 | |
| - Created actors count: 0 | |
| - owners_: 0 | |
| - actor_to_register_callbacks_: 0 | |
| - actor_to_restart_callbacks_: 0 | |
| - actor_to_create_callbacks_: 0 | |
| - sorted_destroyed_actor_list_: 0 | |
| GcsResourceManager: | |
| - GetAllAvailableResources request count: 1 | |
| - GetAllTotalResources request count: 0 | |
| - GetAllResourceUsage request count: 2 | |
| GcsPlacementGroupManager: | |
| - CreatePlacementGroup request count: 0 | |
| - RemovePlacementGroup request count: 0 | |
| - GetPlacementGroup request count: 0 | |
| - GetAllPlacementGroup request count: 0 | |
| - WaitPlacementGroupUntilReady request count: 0 | |
| - GetNamedPlacementGroup request count: 0 | |
| - Scheduling pending placement group count: 0 | |
| - Registered placement groups count: 0 | |
| - Named placement group count: 0 | |
| - Pending placement groups count: 0 | |
| - Infeasible placement groups count: 0 | |
| Publisher: | |
| RAY_LOG_CHANNEL | |
| - cumulative published messages: 8 | |
| - cumulative published bytes: 5596 | |
| - current buffered bytes: 252 | |
| GCS_NODE_INFO_CHANNEL | |
| - cumulative published messages: 2 | |
| - cumulative published bytes: 638 | |
| - current buffered bytes: 100 | |
| GCS_WORKER_DELTA_CHANNEL | |
| - cumulative published messages: 25 | |
| - cumulative published bytes: 2425 | |
| - current buffered bytes: 97 | |
| GCS_JOB_CHANNEL | |
| - cumulative published messages: 2 | |
| - cumulative published bytes: 499 | |
| - current buffered bytes: 254 | |
| [runtime env manager] ID to URIs table: | |
| [runtime env manager] URIs reference table: | |
| GcsTaskManager: | |
| -Total num task events reported: 24 | |
| -Total num status task events dropped: 0 | |
| -Total num profile events dropped: 0 | |
| -Current num of task events stored: 7 | |
| -Total num of actor creation tasks: 0 | |
| -Total num of actor tasks: 0 | |
| -Total num of normal tasks: 6 | |
| -Total num of driver tasks: 1 | |
| GcsAutoscalerStateManager: | |
| - last_seen_autoscaler_state_version_: 0 | |
| - last_cluster_resource_state_version_: 0 | |
| - pending demands: | |
| Global stats: 703 total (4 active) | |
| Queueing time: mean = 3.426 ms, max = 262.603 ms, min = 688.000 ns, total = 2.408 s | |
| Execution time: mean = 640.951 us, total = 450.589 ms | |
| Event stats: | |
| GcsInMemoryStore.Get - 101 total (0 active), Execution time: mean = 60.733 us, total = 6.134 ms, Queueing time: mean = 24.921 us, max = 163.039 us, min = 2.876 us, total = 2.517 ms | |
| GcsInMemoryStore.Put - 94 total (0 active), Execution time: mean = 2.828 ms, total = 265.865 ms, Queueing time: mean = 19.480 ms, max = 261.747 ms, min = 3.096 us, total = 1.831 s | |
| InternalKVGcsService.grpc_server.InternalKVGet - 74 total (0 active), Execution time: mean = 627.482 us, total = 46.434 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVGet.HandleRequestImpl - 74 total (0 active), Execution time: mean = 42.858 us, total = 3.172 ms, Queueing time: mean = 101.472 us, max = 211.289 us, min = 9.108 us, total = 7.509 ms | |
| event_loop_lag_probe - 42 total (0 active), Execution time: mean = 23.078 us, total = 969.268 us, Queueing time: mean = 360.978 us, max = 14.441 ms, min = 4.171 us, total = 15.161 ms | |
| NodeInfoGcsService.grpc_server.GetAllNodeInfo - 38 total (0 active), Execution time: mean = 557.927 us, total = 21.201 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.GetAllNodeInfo.HandleRequestImpl - 38 total (0 active), Execution time: mean = 38.393 us, total = 1.459 ms, Queueing time: mean = 96.796 us, max = 178.739 us, min = 7.285 us, total = 3.678 ms | |
| WorkerInfoGcsService.grpc_server.ReportWorkerFailure.HandleRequestImpl - 25 total (0 active), Execution time: mean = 52.511 us, total = 1.313 ms, Queueing time: mean = 105.784 us, max = 236.517 us, min = 6.337 us, total = 2.645 ms | |
| WorkerInfoGcsService.grpc_server.AddWorkerInfo - 25 total (0 active), Execution time: mean = 617.987 us, total = 15.450 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| WorkerInfoGcsService.grpc_server.AddWorkerInfo.HandleRequestImpl - 25 total (0 active), Execution time: mean = 34.653 us, total = 866.336 us, Queueing time: mean = 111.792 us, max = 216.258 us, min = 21.151 us, total = 2.795 ms | |
| WorkerInfoGcsService.grpc_server.ReportWorkerFailure - 25 total (0 active), Execution time: mean = 904.706 us, total = 22.618 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVPut - 19 total (0 active), Execution time: mean = 680.195 us, total = 12.924 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVPut.HandleRequestImpl - 19 total (0 active), Execution time: mean = 43.385 us, total = 824.308 us, Queueing time: mean = 119.776 us, max = 263.182 us, min = 90.168 us, total = 2.276 ms | |
| GcsResourceManager::Update - 12 total (0 active), Execution time: mean = 81.412 us, total = 976.947 us, Queueing time: mean = 124.367 us, max = 183.919 us, min = 30.847 us, total = 1.492 ms | |
| RayletLoadPulled - 10 total (1 active), Execution time: mean = 274.731 us, total = 2.747 ms, Queueing time: mean = 61.038 us, max = 93.825 us, min = 21.010 us, total = 610.385 us | |
| NodeManagerService.grpc_client.GetResourceLoad.OnReplyReceived - 7 total (0 active), Execution time: mean = 79.144 us, total = 554.010 us, Queueing time: mean = 107.524 us, max = 259.712 us, min = 24.485 us, total = 752.667 us | |
| GcsInMemoryStore.GetAll - 7 total (0 active), Execution time: mean = 21.490 us, total = 150.431 us, Queueing time: mean = 93.809 us, max = 124.265 us, min = 13.134 us, total = 656.663 us | |
| NodeManagerService.grpc_client.GetResourceLoad - 7 total (0 active), Execution time: mean = 2.369 ms, total = 16.586 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.GetClusterId - 5 total (0 active), Execution time: mean = 2.965 ms, total = 14.826 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.GetClusterId.HandleRequestImpl - 5 total (0 active), Execution time: mean = 437.583 us, total = 2.188 ms, Queueing time: mean = 981.072 us, max = 4.438 ms, min = 81.993 us, total = 4.905 ms | |
| ClusterResourceManager.ResetRemoteNodeView - 4 total (1 active), Execution time: mean = 7.766 us, total = 31.066 us, Queueing time: mean = 64.979 us, max = 132.367 us, min = 56.678 us, total = 259.917 us | |
| NodeInfoGcsService.grpc_server.CheckAlive - 4 total (0 active), Execution time: mean = 573.886 us, total = 2.296 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.CheckAlive.HandleRequestImpl - 4 total (0 active), Execution time: mean = 36.769 us, total = 147.074 us, Queueing time: mean = 112.926 us, max = 121.067 us, min = 105.672 us, total = 451.704 us | |
| PeriodicalRunner.RunFnPeriodically - 4 total (0 active), Execution time: mean = 186.255 us, total = 745.018 us, Queueing time: mean = 132.412 ms, max = 262.603 ms, min = 2.112 ms, total = 529.648 ms | |
| GcsInMemoryStore.Keys - 3 total (0 active), Execution time: mean = 9.717 us, total = 29.151 us, Queueing time: mean = 62.048 us, max = 69.223 us, min = 50.973 us, total = 186.143 us | |
| NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage.HandleRequestImpl - 2 total (0 active), Execution time: mean = 74.737 us, total = 149.474 us, Queueing time: mean = 131.979 us, max = 151.458 us, min = 112.500 us, total = 263.958 us | |
| NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage - 2 total (0 active), Execution time: mean = 773.086 us, total = 1.546 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVDel - 1 total (0 active), Execution time: mean = 867.026 us, total = 867.026 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVExists - 1 total (0 active), Execution time: mean = 598.827 us, total = 598.827 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.RegisterNode.HandleRequestImpl - 1 total (0 active), Execution time: mean = 180.775 us, total = 180.775 us, Queueing time: mean = 134.900 us, max = 134.900 us, min = 134.900 us, total = 134.900 us | |
| HealthCheck - 1 total (0 active), Execution time: mean = 8.534 us, total = 8.534 us, Queueing time: mean = 161.376 us, max = 161.376 us, min = 161.376 us, total = 161.376 us | |
| GcsHealthCheckManager::RemoveNode - 1 total (0 active), Execution time: mean = 1.065 us, total = 1.065 us, Queueing time: mean = 688.000 ns, max = 688.000 ns, min = 688.000 ns, total = 688.000 ns | |
| NodeInfoGcsService.grpc_server.UnregisterNode - 1 total (0 active), Execution time: mean = 885.927 us, total = 885.927 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetAllJobInfo.HandleRequestImpl - 1 total (0 active), Execution time: mean = 29.314 us, total = 29.314 us, Queueing time: mean = 138.095 us, max = 138.095 us, min = 138.095 us, total = 138.095 us | |
| JobInfoGcsService.grpc_server.MarkJobFinished.HandleRequestImpl - 1 total (0 active), Execution time: mean = 11.970 us, total = 11.970 us, Queueing time: mean = 13.327 us, max = 13.327 us, min = 13.327 us, total = 13.327 us | |
| JobInfoGcsService.grpc_server.MarkJobFinished - 1 total (0 active), Execution time: mean = 466.188 us, total = 466.188 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GCSServer.deadline_timer.debug_state_dump - 1 total (1 active, 1 running), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GcsInMemoryStore.BatchDelete - 1 total (0 active), Execution time: mean = 3.237 us, total = 3.237 us, Queueing time: mean = 15.730 us, max = 15.730 us, min = 15.730 us, total = 15.730 us | |
| JobInfoGcsService.grpc_server.AddJob.HandleRequestImpl - 1 total (0 active), Execution time: mean = 171.609 us, total = 171.609 us, Queueing time: mean = 103.440 us, max = 103.440 us, min = 103.440 us, total = 103.440 us | |
| InternalKVGcsService.grpc_server.InternalKVExists.HandleRequestImpl - 1 total (0 active), Execution time: mean = 42.840 us, total = 42.840 us, Queueing time: mean = 102.345 us, max = 102.345 us, min = 102.345 us, total = 102.345 us | |
| GcsInMemoryStore.Delete - 1 total (0 active), Execution time: mean = 22.549 us, total = 22.549 us, Queueing time: mean = 12.027 us, max = 12.027 us, min = 12.027 us, total = 12.027 us | |
| GcsHealthCheckManager::AddNode - 1 total (0 active), Execution time: mean = 16.304 us, total = 16.304 us, Queueing time: mean = 744.000 ns, max = 744.000 ns, min = 744.000 ns, total = 744.000 ns | |
| NodeResourceInfoGcsService.grpc_server.GetAllAvailableResources - 1 total (0 active), Execution time: mean = 651.603 us, total = 651.603 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.AddJob - 1 total (0 active), Execution time: mean = 781.282 us, total = 781.282 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetNextJobID.HandleRequestImpl - 1 total (0 active), Execution time: mean = 32.127 us, total = 32.127 us, Queueing time: mean = 139.244 us, max = 139.244 us, min = 139.244 us, total = 139.244 us | |
| NodeResourceInfoGcsService.grpc_server.GetAllAvailableResources.HandleRequestImpl - 1 total (0 active), Execution time: mean = 53.296 us, total = 53.296 us, Queueing time: mean = 97.632 us, max = 97.632 us, min = 97.632 us, total = 97.632 us | |
| NodeInfoGcsService.grpc_server.UnregisterNode.HandleRequestImpl - 1 total (0 active), Execution time: mean = 393.774 us, total = 393.774 us, Queueing time: mean = 95.557 us, max = 95.557 us, min = 95.557 us, total = 95.557 us | |
| GcsInMemoryStore.Exists - 1 total (0 active), Execution time: mean = 15.993 us, total = 15.993 us, Queueing time: mean = 12.340 us, max = 12.340 us, min = 12.340 us, total = 12.340 us | |
| InternalKVGcsService.grpc_server.GetInternalConfig - 1 total (0 active), Execution time: mean = 708.710 us, total = 708.710 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.RegisterNode - 1 total (0 active), Execution time: mean = 1.201 ms, total = 1.201 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetAllJobInfo - 1 total (0 active), Execution time: mean = 593.685 us, total = 593.685 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetNextJobID - 1 total (0 active), Execution time: mean = 544.448 us, total = 544.448 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GCSServer.deadline_timer.debug_state_event_stats_print - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVDel.HandleRequestImpl - 1 total (0 active), Execution time: mean = 72.375 us, total = 72.375 us, Queueing time: mean = 111.929 us, max = 111.929 us, min = 111.929 us, total = 111.929 us | |
| InternalKVGcsService.grpc_server.GetInternalConfig.HandleRequestImpl - 1 total (0 active), Execution time: mean = 34.576 us, total = 34.576 us, Queueing time: mean = 163.583 us, max = 163.583 us, min = 163.583 us, total = 163.583 us |