You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Do not start nested parallel work from inside a thread-server job - #6260
Threaded level-2 drivers call those kernels from inside each worker job.
Affected level-2 callers on columns longer than 10,000 elements (DOT):
dgbmv, zgbmv (and sgbmv, cgbmv on arm64) with trans = 'T' or 'C'
dtbmv, ztbmv (and stbmv, ctbmv on arm64) with trans = 'T' or 'C'
dtpmv, ztpmv (and stpmv, ctpmv on arm64) with trans = 'T' or 'C'
zhpmv (and chpmv on arm64)
Affected level-2 callers on strided vectors longer than 10,000 elements on arm64 (COPY_K):
sger, dger, cgeru, cgerc, zgeru, zgerc
strmv, dtrmv, ctrmv, ztrmv
stbmv, dtbmv, ctbmv, ztbmv
stpmv, dtpmv, ctpmv, ztpmv
ssyr, dsyr, csyr, zsyr, cher, zher
ssyr2, dsyr2, csyr2, zsyr2, cher2, zher2
sspr, dspr, cspr, zspr, chpr, zhpr
sspr2, dspr2, cspr2, zspr2, chpr2, zhpr2
In pthreads builds num_cpu_avail() had no nesting check, so each worker re-entered exec_blas(). Once all pool threads were occupied by the outer call, exec_blas_async() spun forever waiting for idle worker slots while holding exec_queue_lock before signalling any assigned sub-jobs.
This PR tracks whether the current thread is executing a thread-server job and makes num_cpu_avail() return 1 when set. Covered execution paths:
pool workers in the pthreads and win32 servers
exec_threads() for OpenMP and registered threads callbacks
the calling thread while it runs its own share in exec_blas()
Tests
utest/test_nested_thread.c covers:
dgbmv('T'), sgbmv('T'), zgbmv('C') with column DOT > 10,000 elements
dger, dtbmv with strided COPY > 10,000 elements
With the POSIX pthreads server each case runs in a child process under a timeout so a regression fails cleanly instead of hanging CI.
Verification (before → after on develop)
x86_64 (Zen, Haswell, Sandybridge, SkylakeX under Intel SDE): dgbmv('T'), zgbmv('C'), zhpmv, dtpmv('T') hang → pass (170/170openblas_utest, 1545/1545openblas_utest_ext)
arm64 (NEOVERSEN1 under qemu): all 5 nested_thread cases hang → pass (166/166openblas_utest)
USE_OPENMP=1, USE_THREAD=0, and CMake BUILD_SHARED_LIBS=ON builds: all pass
Several level-1 kernels split vectors longer than 10,000 elements across
num_cpu_avail(1)threads.Affected level-1 kernels:
kernel/x86_64/ddot.c,kernel/x86_64/zdot.c(Haswell, Zen, SkylakeX/Cooperlake/Sapphire Rapids, Sandybridge, Bulldozer family)kernel/arm64/dot.c,kernel/arm64/zdot_thunderx2t99.c,kernel/arm64/copy_thunderx2t99.c(Neoverse N1/N2/V1/V2, SVE, SME, A64FX, Vortex, ThunderX2/3)Threaded level-2 drivers call those kernels from inside each worker job.
Affected level-2 callers on columns longer than 10,000 elements (
DOT):dgbmv,zgbmv(andsgbmv,cgbmvon arm64) withtrans = 'T'or'C'dtbmv,ztbmv(andstbmv,ctbmvon arm64) withtrans = 'T'or'C'dtpmv,ztpmv(andstpmv,ctpmvon arm64) withtrans = 'T'or'C'zhpmv(andchpmvon arm64)Affected level-2 callers on strided vectors longer than 10,000 elements on arm64 (
COPY_K):sger,dger,cgeru,cgerc,zgeru,zgercstrmv,dtrmv,ctrmv,ztrmvstbmv,dtbmv,ctbmv,ztbmvstpmv,dtpmv,ctpmv,ztpmvssyr,dsyr,csyr,zsyr,cher,zherssyr2,dsyr2,csyr2,zsyr2,cher2,zher2sspr,dspr,cspr,zspr,chpr,zhprsspr2,dspr2,cspr2,zspr2,chpr2,zhpr2In pthreads builds
num_cpu_avail()had no nesting check, so each worker re-enteredexec_blas(). Once all pool threads were occupied by the outer call,exec_blas_async()spun forever waiting for idle worker slots while holdingexec_queue_lockbefore signalling any assigned sub-jobs.This PR tracks whether the current thread is executing a thread-server job and makes
num_cpu_avail()return1when set. Covered execution paths:exec_threads()for OpenMP and registered threads callbacksexec_blas()Tests
utest/test_nested_thread.ccovers:dgbmv('T'),sgbmv('T'),zgbmv('C')with columnDOT> 10,000 elementsdger,dtbmvwith stridedCOPY> 10,000 elementsWith the POSIX pthreads server each case runs in a child process under a timeout so a regression fails cleanly instead of hanging CI.
Verification (before → after on
develop)dgbmv('T'),zgbmv('C'),zhpmv,dtpmv('T')hang → pass (170/170openblas_utest,1545/1545openblas_utest_ext)NEOVERSEN1under qemu): all 5nested_threadcases hang → pass (166/166openblas_utest)USE_OPENMP=1,USE_THREAD=0, and CMakeBUILD_SHARED_LIBS=ONbuilds: all pass