Content
previous page: Scaling behavior of psana1 det.calib method in multicore processing with mpi
2024-02-06 Test of milano216 host with perf
Description
Using command:
perf stat -e cache-references,cache-misses,cycles,instructions,branches,branch-misses,faults,migrations,page-faults,L1-dcache-load-misses,L1-icache-load-misses python test-scaling-subproc.py <parameter>
where parameter defines test for different number of CPUs, e.g. <parameter> = −1,−2,−8,−13,−16,−17,−18 stands for test on single, 8, 16, 32, 56, 64, 128 CPUs.
Results
Summary
number of CPU | cache- references | cache- misses | cycles | instructions | branches | branch- misses | faults | page-faults | L1-dcache- load-misses | L1-icache- load-misses | L1-icache Ratio N/1 | cmt |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 4,522,410,200 | 112,207,635 | 224,402,878,245 | 428,582,543,872 | 59,430,436,824 | 2,353,206,592 | 657,277 | 657,277 | 2,169,783,808 | 7,173,374 | 1 | |
8 | 35,293,654,947 | 675,772,563 | 18,710,029,709 | 17,164,781,068 | 42,407,266 | 5.9 | ||||||
16 | 71,125,012,043 | 2,509,743,885 | 37,401,077,277 | 34,764,585,133 | 82,908,203 | 11.6 | ||||||
32 | 140,229,421,945 | 5,022,345,750 | 74,783,808,615 | 68,615,480,748 | 163,094,161 | 22.7 | ||||||
56 | 245,664,589,385 | 5,986,128,102 | 130,897,170,304 | 119,933,873,577 | 288,403,921 | 40.2 | ||||||
64 | 281,639,175,978 | 8,968,404,974 | 149,569,155,086 | 137,584,278,754 | 330,750,296 | 46.1 | ||||||
120 | 532,229,037,371 | 14,227,944,434 | 29,404,359,241,173 | 51,095,884,028,391 | 7,053,547,766,317 | 280,479,284,507 | 73,250,012 | 73,250,012 | 260,078,672,869 | 618,858,635 | 86.2 | |
2024-02-07 Test of milano216 host with command perf
Description
Running perf with mpirun on a single and 80 CPUs:
perf stat -e cache-references,cache-misses,cycles,instructions,branches,branch-misses,faults,migrations,page-faults,L1-dcache-load-misses,L1-icache-load-misses,dTLB-load-misses,iTLB-load-misses mpirun -n 1 python Detector/examples/test-scaling-mpi.py
perf stat -e cache-references,cache-misses,cycles,instructions,branches,branch-misses,faults,migrations,page-faults,L1-dcache-load-misses,L1-icache-load-misses,dTLB-load-misses,iTLB-load-misses mpirun -n 80 python Detector/examples/test-scaling-mpi.py
Results
Summary
number of mpi cores | cache- references | cache- misses | cycles | instructions | branches | branch- misses | faults | page-faults | L1-dcache- load-misses | L1-icache- load-misses | dTLB- load-misses | iTLB- load-misses | cmt |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 4,448,830,552 | 90,374,312 | 222,814,516,280 | 426,700,282,993 | 58,876,394,584 | 2,343,687,188 | 635,183 | 635,183 | 2,158,358,417 | 5,694,036 | 4,282,821 | 890,671 | |
80 | 349,526,509,383 | 5,932,480,814 | 18,768,444,974,036 | 33,983,153,714,284 | 4,684,730,635,234 | 186,649,297,019 | 52,121,421 | 52,121,421 | 171,500,392,922 | 267,672,856 | 339,145,247 | 69,780,394 | |
Ratio (80)/(1) | 79.4 | 65.7 | 84.1 | 79.6 | 79.5 | 79.7 | 82.0 | 82.0 | 79.3 | 47.0 | 79.2 | 78.4 |
2024-02-09 Test of milano216 host with command perf
Use commands with changed list of counters like
perf stat -e stalled-cycles-backend,stalled-cycles-frontend,ls_l1_d_tlb_miss.all,l1_dtlb_misses,l1_data_cache_fills_all,bp_l1_tlb_miss_l2_tlb_miss.if2m,bp_l1_tlb_miss_l2_tlb_miss,l2_dtlb_misses,l2_itlb_misses python test-scaling-subproc.py -8
Summary
number of CPU | stalled-cycles-backend | stalled-cycles-frontend | ls_l1_d_tlb_miss.all | l1_dtlb_misses | l1_data_cache_fills_all | bp_l1_tlb_miss_l2_tlb_miss.if2m | bp_l1_tlb_miss_l2_tlb_miss | l2_dtlb_misses | l2_itlb_misses | |
---|---|---|---|---|---|---|---|---|---|---|
1 | 143828614 | 230987724 | 33227437 | 32845193 | 2179469714 | 3701 | 769309 | 4833384 | 719026 | |
8 | 2105881833 | 3421108359 | 172779030 | 173508212 | 18216564874 | 25606 | 6124897 | 31719300 | 5591821 | |
16 | 8796313234 | 8018691890 | 327892753 | 326337183 | 34551341060 | 55331 | 12467976 | 68227221 | 10605352 | |
24 | 10413149941 | 10519490870 | 491673248 | 490566093 | 51539384297 | 78433 | 17889621 | 96922469 | 15177116 | |
32 | 17251055297 | 13858554955 | 671047247 | 666230997 | 68736842168 | 105874 | 23936978 | 135322250 | 21599940 | |
56 | 17892504080 | 24120493158 | 1136778538 | 1135448325 | 120696775952 | 178082 | 42679843 | 234498254 | 38164171 | |
64 | 27304844238 | 27697522017 | 1258999729 | 1258031354 | 141469109046 | 201330 | 50957218 | 258609632 | 43825042 | |
120 | 45388735746 | 46279264661 | 2382065820 | 2376507106 | 264016453328 | 375699 | 93410817 | 488308155 | 78261952 | |