...

• bad single-to-multicore scaling issue has nothing to do with particular algorithm, it is common problem for any algorithm

2024-01-16

...

Test for simulated events

Time consuming code

def time_consuming_algorithm():
    sh2d = (1024,1024)
    sh3d = (3,) + sh2d
    a = random_standard(shape=sh2d, mu=10, sigma=2, dtype=np.float64)
    b = random_standard(shape=sh3d, mu=20, sigma=3, dtype=np.float64)
    gr1 = a>=11
    gr2 = (a>9) & (a<11)
    gr3 = a<=9
    a[gr1] -= b[0, gr1]
    a[gr2] -= b[1, gr2]
    a[gr3] -= b[2, gr3]

Results

Single core processing

Image RemovedImage Removed

80-core processing for ranks 0, 30, and 60

Image RemovedImage Removed

Image RemovedImage Removed

...

2024-01-16 Test Summary

• runing on sdfmilan031 - shows minor ~ 5% the mean time difference between 1-core and 80-core processing is larger

2024-01-17 Test for simulated events with minor modifications

...

def time_consuming_algorithm(a, b):
    gr1 = a>=11
    gr2 = (a>9) & (a<11)
    gr3 = a<=9
    t0_sec = time()
    a[gr1] -= b[0, gr1]
    a[gr2] -= b[1, gr2]
    a[gr3] -= b[2, gr3]
    #med = np.median(a)
    return time() - t0_sec

Results

Single core processing

80-core processing for ranks 0, 30, and 60

2024-01-17 Test Summary

• if random array generation is excluded from time measurement, the mean time difference between 1-core and 80-core processing is larger
• the main contribution to time difference comes from reserved host...
  • sdfmilan047 - looks like extremely "time-noisy" in multi-core processing...
  • sdfmilan031 -  in previous test looks pretty "calm"

...