Page History
...
| host name | core 0/1 | 0/80 | 10/80 | 20/80 | 30/80 | 40/80 | 50/80 | 60/80 | 70/80 | 80-core average | Ratio | Largest ratio |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sdfmilan027 | 192 ±6 | 241 ±19 | 430 ±32 | 336 ±24 | 287 ±16 | 245 ±17 | 412 ±30 | 342 ±24 | 327.6 | 1.71 | 2.24 | |
| sdfmilan049 | 210 ±1 | 280 ±19 | 369 ±28 | 245 ±45 | 282 ±17 | 290 ±24 | 367 ±28 | 242 ±37 | 296.4 | 1.41 | 1.75 | |
| sdfmilan056 | 191 ±3 | 328 ±21 | 269 ±40 | 258 ±32 | 347 ±28 | 324 ±22 | 254 ±34 | 346 ±30 | 303.7 | 1.59 | 1.82 | |
| sdfmilan057 | 190 ±4 | 262 ±20 | 292 ±21 | 439 ±26 | 226 ±12 | 256 ±19 | 297 ±20 | 467 ±36 | 319.9 | 1.68 | 2.46 | |
| sdfmilan065 | 203 ±7 | 281 ±15 | 345 ±23 | 245 ±22 | 248 ±20 | 276 ±15 | 353 ±24 | 250 ±22 | 285.4 | 1.41 | 1.74 | |
| sdfmilan105 | 190 ±2 | 261 ±36 | 277 ±20 | 282 ±15 | 276 ±14 | 263 ±48 | 273 ±19 | 284 ±26 | 273.7 | 1.44 | 1.49 | |
| sdfmilan130 | 190 ±3 | 394 ±26 | 270 ±23 | 323 ±20 | 230 ±10 | 379 ±27 | 276 ±24 | 322 ±20 | 313.4 | 1.65 | 2.07 | |
| sdfmilan202 | 210 ±0 | 346 ±32 | 351 ±28 | 281 ±16 | 286 ±16 | 355 ±34 | 345 ±28 | 297 ±16 | 323.0 | 1.54 | 1.69 | |
| sdfmilan203 | 191 ±4 | 281 ±26 | 355 ±31 | 288 ±26 | 332 ±44 | 271 ±32 | 347 ±26 | 282 ±21 | 308.0 | 1.61 | 1.86 | |
| sdfmilan225 | 204 ±7 | 409 ±29 | 296 ±23 | 230 ±10 | 230 ±10 | 398 ±26 | 289 ±24 | 232 ±9 | 297.7 | 1.46 | 2.00 | |
sdfmilan213 cpu_num | 293 6 0 | 339 ±18 13 | 355 ±22 45 | 605 ±104 74 | 388 ±27 103 | 337 ±18 5 | 361 ±21 44 | 613 ±95 72 | 422 ±27 106 | 427.5 | 1.46 | 2.09 |
sdfmilan047 cpu_num | 210 1 0 | 639 ±34 8 | 483 ±219 32 | 474 ±284 65 | 744 ±75 104 | 637 ±43 40 | 588 ±266 36 | 443 ±202 67 | 722 ±75 110 | 590.8 | 2.81 | 3.54 |
sdfmilan220 cpu_num | 210 1 0 | 649 ±29 7 | 669 ±134 34 | 500 ±214 76 | 1039 ±95 105 | 707 ±50 10 | 654 ±113 40 | 490 ±229 72 | 1040 ±83 104 | 718.5 | 3.42 | 4.95 |
Last 3 lines in table also show cpu_num to prevent contribution from weka nodes
| Code Block | ||||
|---|---|---|---|---|
| ||||
2024-01-19 6:24PM
Hi Mikhail,
I think those code can fetch a physical core number:
>>> import psutil
>>> print(psutil.Process().cpu_num())
3
>>>
Can you add that to your numpy tests to make sure that cores 16-23 aren’t used by your mpi processes? I think Wilko thought those 8 cores run weka filesystem processes.
Thanks, and have a good weekend,
chris |
2024-01-19 Time difference vs array size
...
| host name | core 0/1 | 0/80 | 10/80 | 20/80 | 30/80 | 40/80 | 50/80 | 60/80 | 70/80 | 80-core average | Ratio | Largest ratio |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sdfmilan225 | 766 ±20 | 1251 ±91 | 1099 ±108 | 958 ±174 | 955 ±199 | 1206 ±83 | 1086 ±106 | 949 ±181 | 952 ±231 | 1057 | 1.38 | 1.63 |
| sdfmilan031 | 760 ±1 | 1141 ±81 | 1144 ±94 | 910 ±111 | 947 ±118 | 1221 ±74 | 1157 ±93 | 947 ±108 | 937 ±104 | 1051 | 1.38 | 1.61 |
2024-01-19-22 Test Summary
- weka FS nodes do not show up in processing ranks
- decreasing array size from 4M to 1M decreases time ratio from ~1.5 to 1.15
- increasing array size from 4M to 14M does not increases time ratio...
References
Overview
Content Tools