libdl does not support TLS

[Chris Johns]

The symbol geernator does not create the correct symbols for libdl to use. And libdl does not handle TLS symbols correctly.

There is not backend support for TLS in the arch code.

[Chris Johns]

We need to add support to din the TSL base for a thread for each architecture:

• aarch64
• arm
• bfin
• i386
• lm32
• m68k
• mips
• moxie
• powerpc
• riscv
• sparc
• v850

AARCH64

static void* rtems_rtl_tls_get_base (void) {
  intptr_t value;
  __asm__ volatile (
    \"mrs %0, TPIDR_EL0\" : \"=&r\" ( value ) : : \"memory\"
  );",
  return value;
}

[Chris Johns]

Joel, we voted and you are best to do this as you have all arch tools built.

[Chris Johns]

Test dl11 fails on arm with:

*** BEGIN OF TEST libdl (RTL) 11 ***
*** TEST VERSION: 6.0.0.10da3025186cadf2d78e34cf81cce02121069d44
*** TEST STATE: EXPECTED_PASS
*** TEST BUILD: RTEMS_DEBUG RTEMS_POSIX_API
*** TEST TOOLS: 12.3.1 20230626 (RTEMS 6, RSB 8e568b2ca3489d6bfa48e1d29618ea9b48a5b408, Newlib 4c7d0df)
load: /dl11-o1.o
handle: 0x20e148 loaded
dlsym ptr_call failed: ret value bad

and on aarch64 with:

*** BEGIN OF TEST libdl (RTL) 11 ***
*** TEST VERSION: 6.0.0.10da3025186cadf2d78e34cf81cce02121069d44
*** TEST STATE: EXPECTED_PASS
*** TEST BUILD: RTEMS_DEBUG RTEMS_POSIX_API
*** TEST TOOLS: 12.3.1 20230626 (RTEMS 6, RSB 8e568b2ca3489d6bfa48e1d29618ea9b48a5b408, Newlib 4c7d0df)
load: /dl11-o1.o
handle: 0x40118610 has unresolved externals
dlsym ptr_call failed: ret value bad

The ptr_call asks for a function pointer to:

int *get_errno_ptr (void)
{
  return &errno;
}

and errno is a TLS variable.

[Chris Johns]

I have posted a patch for the general changes need plus dl11 is working for the following architectures:

• aarch64
• arm
• powerpc
• sparc

[Chris Johns]

Remaining archs are:

• bfin
• i386
• lm32
• m68k
• mips
• moxie
• riscv
• v850

I do not know which of these archs support TLS.

[Joel Sherrill]

The following table summarises the state of TLS support for each architecture listed previously. I based this on grep'ing for TLS in the score/cpu port directory and looking at generated assembly.

Architecture	Mechanism	Support in RTEMS
bfin	_emutls_get_address()	No
i386	%gs register when not compiled for PIC	Yes
lm32	emutls_get_address()	No
m68k	m68k_read_tp()	Yes
mips	Illegal Instruction Exception	No
moxie	emutls_get_address()	No
riscv	%tp register	Yes
v850	_emutls_get_address()	No

Question: Some of the architectures have an implementation of tls_get_addr(). Could this be copied and used as emutls_get_addr()?

Question: Do we want to disable use of TLS in the tools for those we have no TLS support for? I think this is the safest option.

I used this program to see what GCC generates.

__thread int i;

int returni(void)
{
  return i;
}

Note: For the MIPS, see gcc/config/mips/mips.md for a description of the illegal instruction used.

[Gedare Bloom]

v850 is slated for deprecation. quite likely so should any of the architectures that don't have TLS

[Chris Johns]

Question: Do we want to disable use of TLS in the tools for those we have no TLS support for? I think this is the safest option.

The change is not adding the ability to load TLS variables. That is out is scope for this change. The regression is only for archs that support TLS and where newlib has been updated to use TLS for reeent data. If this newlib has not been been updated there is no regression so I do not think we need to disable archs.

Note:

1. If an arch supports TLS and newlib has not been updated and a user attempts to load a TLS variable an error is generated which is the same error as before.
2. The rtems_rtl_tls_get_base() call needs to be present if the base image has TSL symbols. If it does not there should be no references.

[Chris Johns]

Question: Some of the architectures have an implementation of tls_get_addr(). Could this be copied and used as emutls_get_addr()?

If there is a call that can be made then it can be used.

[Chris Johns]

v850 is slated for deprecation. quite likely so should any of the architectures that don't have TLS

The rtrms_rtl_tls_get_base() call can return NULL for this arch. An active user can update or ask to have the code updated. Is the compiler defined __v850__?

[Chris Johns]

Test results

This is the test results for the ones I can simulate using the v2 patch.

AARCH64

Failures:

• dl06.exe
• dl09.exe

ARM

Failures:

• dl06.exe
• dl09.exe

PowerPC

Failures:

• dl06.exe
• dl09.exe

SPARC

Expected Fail:

• dl06.exe

[Chris Johns <chrisj@…>]

In [changeset:"aba4d3b8daa9fbb63dff9e1f99fe76572186ef77/rtems-tools" aba4d3b/rtems-tools]:

linker: Add TLS support to the symbol table generator

Updates #4920

[Chris Johns <chrisj@…>]

In [changeset:"27da374e481b0067896f70b21c9d89c04f87a101/rtems" 27da374/rtems]:

libdl: Add support to import base image TLS symbols

This change requires an rtems-tools update for symbol generation.

Working architectures:

• aarch64
• arm
• powerpc
• sparc

No newlib TLS support but checked:

• i386
• m69k

Updates #4920

[Chris Johns <chrisj@…>]

In [changeset:"0860f9948dec7c001391948337723b313c052d1b/rtems-source-builder" 0860f99/rtems-source-builder]:

rtems/tools: Pick up the rtems-syms and rtems-test sis changes

Updates #4920

[Kinsey Moore]

With the update from Chris above, that leaves these architectures to check:

• bfin
• lm32
• mips
• moxie
• riscv
• v850

[Chris Johns]

I think v850 can be left if it builds as it is to be removed once we branch 6 for release. For the others knowing which has TLS support enabled would be a good start.

[Sebastian Huber <sebastian.huber@…>]

In [changeset:"49a88a73a871d23bb7129a5048e4b5331527e84a/rtems" 49a88a7/rtems]:

score: Add _CPU_Get_TLS_thread_pointer()

Add _CPU_Get_TLS_thread_pointer() to get the thread pointer which is
used to get the address of thread-local storage objects associated with
a thread.

Update #4920.

[Sebastian Huber <sebastian.huber@…>]

In [changeset:"de25f012b76978464a94ffed603d6a6bb79b744e/rtems" de25f012/rtems]:

libdl: Use _CPU_Get_TLS_thread_pointer()

Update #4920.

[Kinsey Moore]

Remaining items on the list here are:

• bfin
• lm32
• mips
• moxie

I'm building tools to check these.

[Kinsey Moore]

For blackfin in sptls01.exe:

Retrieving the TLS space:

331c: ff e3 22 f5 CALL 0x1d60 <_emutls_get_address>;
3320: 20 32 P4 = R0;

Retrieving the value for the only TLS item in this test:

3322: 60 99 R0 = B[P4] (X);
3324: 00 43 R0 = R0.B (X);

Setting the value for the only TLS item in this test to 5:

3340: 28 60 R0 = 0x5 (X); /* R0=0x5( 5) */
3342: 20 9b B[P4] = R0;

[Kinsey Moore]

For LM32 in sptls01.exe:

Retrieving TLS space:

8000498: f8 00 24 2f calli 8009554 <emutls_get_address>

Retrieving the value:

80004a0: 40 22 00 00 lbu r2,(r1+0)

Storing the value:

80004cc: 31 81 00 00 sb (r12+0),r1 (there was a "mv r12,r1" in the interim)

[Kinsey Moore]

For MIPS in sptls01.exe:

Retrieving TLS space:
80010334: 7c03e83b rdhwr v0,$29

Retrieving value:
80010344: 90429000 lbu v0,-28672(v0) (negative value is a sign-extended 0x9000, significance not understood)

Storing value:
800106bc: 24030005 li v1,5
800106c0: a0439000 sb v1,-28672(v0)

[Kinsey Moore]

For Moxie in sptls01.exe:

Retrieving TLS space:

2022: 03 00 00 00 jsra 1c4e <emutls_get_address>
2026: 1c 4e

(return value stored in r0)

Retrieving value:

1e98: 1c 32 ld.b $r1, ($r0)

Storing value (TLS address moved to r10):

202a: 1b 20 00 00 ldi.b $r0, 0x5
202e: 00 05
2030: 1e c2 st.b ($r10), $r0

[Sebastian Huber <sebastian.huber@…>]

In [changeset:"a4e1b7fff87f409497595c04f6aa10b860e6cbfe/rtems-tools" a4e1b7f/rtems-tools]:

linker: Fix typo in symbol table generator

Updates #4920.

[Joel Sherrill]

The rdhwr instruction is undefined and should generate an exception. It is expected that the OS will provide the TLS code in that exception. See ​https://www.linux-mips.org/wiki/NPTL for some details.

I have hoped there was a way in GCC to switch to the generic subroutine call but I've never tripped across it. Perhaps something might be obvious if you search for that instruction in the mips specific GCC code.

Replying to Kinsey Moore:

For MIPS in sptls01.exe:

Retrieving TLS space:
80010334: 7c03e83b rdhwr v0,$29

Retrieving value:
80010344: 90429000 lbu v0,-28672(v0) (negative value is a sign-extended 0x9000, significance not understood)

Storing value:
800106bc: 24030005 li v1,5
800106c0: a0439000 sb v1,-28672(v0)

[Kinsey Moore]

The MIPS CPU port in RTEMS doesn't yet support it, but this would be a good application of RTEMS_EXCEPTION_EXTENSIONS to accommodate the existing mechanism.

Replying to Joel Sherrill:

The rdhwr instruction is undefined and should generate an exception. It is expected that the OS will provide the TLS code in that exception. See ​https://www.linux-mips.org/wiki/NPTL for some details.

I have hoped there was a way in GCC to switch to the generic subroutine call but I've never tripped across it. Perhaps something might be obvious if you search for that instruction in the mips specific GCC code.

[Kinsey Moore]

It's on my tasklist to update this issue with the current status.

[Kinsey Moore]

Status update:

bfin, lm32, moxie:
Generated code uses emutls_get_address() to get the TLS base address. This is not yet implemented by _CPU_Get_TLS_thread_pointer().

mips:
Generated code uses the rdhwr instruction. This instruction is not yet used by _CPU_Get_TLS_thread_pointer() and, even if it were, RTEMS does not support it. There is no known way to prevent gcc from using this instruction to retrieve the TLS base address. This requires emulation of this instruction by RTEMS kernel code, possibly using RTEMS_EXCEPTION_EXENSIONS which are not yet available for this platform. There are at least 3 subtasks required to get this working on mips:

• usage of rdhwr in _CPU_Get_TLS_thread_pointer()
• implementation of RTEMS_EXCEPTION_EXENSIONS or equivalent
• usage of the above to emulate rdhwr

v850:
Slated for removal as stated above, not considered.

or1k, sh, sparc64, x86_64:
These do not support dynamic loading and are not being considered for this ticket.

aarch64, arm, i386, m68k, microblaze, nios2, powerpc, riscv, sparc:
_CPU_Get_TLS_thread_pointer() is implemented and expected to function correctly.