4.0.1
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
crp-order-quantity-kpi-service
application UI
Details for Stage 190260 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
13 s
Locality Level Summary:
Process local: 30
Input Size / Records:
9.3 MiB / 60
Output Size / Records:
2.3 MiB / 351756
Associated Job Ids:
52586
DAG Visualization
digraph G { id="graph_190260"; subgraph graph_stage_190260 { id="graph_stage_190260"; isCluster="true"; label="Stage 190260"; subgraph cluster_1107151 { id="cluster_1107151"; isCluster="true"; label="Exchange"; 423538 [id="node_423538" labelType="html" label="ShuffledRowRDD [423538] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; } subgraph cluster_1107148 { id="cluster_1107148"; isCluster="true"; label="WholeStageCodegen (2)"; 423539 [id="node_423539" labelType="html" label="MapPartitionsRDD [423539] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; } subgraph cluster_1107161 { id="cluster_1107161"; isCluster="true"; label="Exchange"; 423545 [id="node_423545" labelType="html" label="ShuffledRowRDD [423545] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; } subgraph cluster_1107157 { id="cluster_1107157"; isCluster="true"; label="WholeStageCodegen (4)"; 423546 [id="node_423546" labelType="html" label="MapPartitionsRDD [423546] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; } subgraph cluster_1107144 { id="cluster_1107144"; isCluster="true"; label="WholeStageCodegen (5)"; 423547 [id="node_423547" labelType="html" label="ZippedPartitionsRDD2 [423547] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; 423548 [id="node_423548" labelType="html" label="MapPartitionsRDD [423548] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; } subgraph cluster_1107143 { id="cluster_1107143"; isCluster="true"; label="InMemoryTableScan"; 423549 [id="node_423549" labelType="html" label="MapPartitionsRDD [423549] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; 423550 [id="node_423550" labelType="html" label="*(5) Project [shardId#6552497, worklistShardItemId#6552498L, version#6832302, qty#6831831, demandChannel#6831811, demandStream#6831812, kpis#6831840]\n+- *(5) SortMergeJoin [shardId#6552497, worklistShardItemId#6552498L], [shardId#6832307, worklistShardItemId#6832308L], Inner\n :- *(2) Sort [shardId#6552497 ASC NULLS FIRST, worklistShardItemId#6552498L ASC NULLS FIRST], false, 0\n : +- Exchange hashpartitioning(shardId#6552497, worklistShardItemId#6552498L, 30), ENSURE_REQUIREMENTS, [plan_id=29052428]\n : +- *(1) Filter (isnotnull(shardId#6552497) AND isnotnull(worklistShardItemId#6552498L))\n : +- InMemoryTableScan [shardId#6552497, worklistShardItemId#6552498L, qty#6831831, demandChannel#6831811, demandStream#6831812, kpis#6831840], [isnotnull(shardId#6552497), isnotnull(worklistShardItemId#6552498L)]\n : +- InMemoryRelation [shardId#6552497, worklistShardItemId#6552498L, qty#6831831, demandChannel#6831811, demandStream#6831812, kpis#6831840], StorageLevel(disk, memory, 2 rep... [423550] [Cached] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; } subgraph cluster_1107197 { id="cluster_1107197"; isCluster="true"; label="InMemoryTableScan"; 423559 [id="node_423559" labelType="html" label="MapPartitionsRDD [423559] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; 423560 [id="node_423560" labelType="html" label="MapPartitionsRDD [423560] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; 423561 [id="node_423561" labelType="html" label="MapPartitionsRDD [423561] [Unordered]<br>start at FileStorageAdapterImpl.java:179}"]; } } 423538->423539; 423545->423546; 423539->423547; 423546->423547; 423547->423548; 423548->423549; 423549->423550; 423550->423559; 423559->423560; 423560->423561; }
423537,423538
423544,423545
423550
Event Timeline
Enable zooming
Tasks: 30. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time