#3740 assigned defect

Libld does not load incrementially linked object file

Reported by: Chris Johns Owned by: Chris Johns
Priority: normal Milestone: 6.1
Component: lib/dl Version: 5
Severity: normal Keywords:
Cc: Blocked By:
Blocking:

Description

If separate object files have the same local symbol and are incrementally linked libdl only uses the first symbol found.

The issue has been raised by Peter in a post to devel:

https://lists.rtems.org/pipermail/devel/2019-April/025623.html

The issue can be shown with:

$ cat foo.c
#include <stdio.h>

extern void foo(void);
void foo(void)
{
    printf("FOO %s\n", "Foo 1");
}
$ cat bar.c
#include <stdio.h>

extern void bar(void);
void bar(void)
{
    printf("BAR %s\n", "Bar 1");
}
$ arm-rtems5-gcc -march=armv7-a -mthumb -c foo.c
$ arm-rtems5-gcc -march=armv7-a -mthumb -c bar.c
$ arm-rtems5-ld -r -o fubar bar.o foo.o
$ arm-rtems5-readelf -s fubar

Symbol table '.symtab' contains 20 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 00000000     0 SECTION LOCAL  DEFAULT    1 
     2: 00000000     0 SECTION LOCAL  DEFAULT    3 
     3: 00000000     0 SECTION LOCAL  DEFAULT    4 
     4: 00000000     0 SECTION LOCAL  DEFAULT    5 
     5: 00000000     0 SECTION LOCAL  DEFAULT    6 
     6: 00000000     0 SECTION LOCAL  DEFAULT    7 
     7: 00000000     0 FILE    LOCAL  DEFAULT  ABS bar.c
     8: 00000000     0 NOTYPE  LOCAL  DEFAULT    3 $d
     9: 00000000     0 NOTYPE  LOCAL  DEFAULT    3 .LC0
    10: 00000008     0 NOTYPE  LOCAL  DEFAULT    3 .LC1
    11: 00000000     0 NOTYPE  LOCAL  DEFAULT    1 $t
    12: 00000000     0 FILE    LOCAL  DEFAULT  ABS foo.c
    13: 00000010     0 NOTYPE  LOCAL  DEFAULT    3 $d
    14: 00000010     0 NOTYPE  LOCAL  DEFAULT    3 .LC0
    15: 00000018     0 NOTYPE  LOCAL  DEFAULT    3 .LC1
    16: 0000001c     0 NOTYPE  LOCAL  DEFAULT    1 $t
    17: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND printf
    18: 0000001d    28 FUNC    GLOBAL DEFAULT    1 foo
    19: 00000001    28 FUNC    GLOBAL DEFAULT    1 bar
$ arm-rtems5-objdump --disassemble --source fubar

fubar:     file format elf32-littlearm

Disassembly of section .text:

00000000 <bar>:
   0:	b580      	push	{r7, lr}
   2:	af00      	add	r7, sp, #0
   4:	f240 0100 	movw	r1, #0
   8:	f2c0 0100 	movt	r1, #0
   c:	f240 0000 	movw	r0, #0
  10:	f2c0 0000 	movt	r0, #0
  14:	f7ff fffe 	bl	0 <printf>
  18:	bf00      	nop
  1a:	bd80      	pop	{r7, pc}

0000001c <foo>:
  1c:	b580      	push	{r7, lr}
  1e:	af00      	add	r7, sp, #0
  20:	f240 0100 	movw	r1, #0
  24:	f2c0 0100 	movt	r1, #0
  28:	f240 0000 	movw	r0, #0
  2c:	f2c0 0000 	movt	r0, #0
  30:	f7ff fffe 	bl	0 <printf>
  34:	bf00      	nop
  36:	bd80      	pop	{r7, pc}

Attachments (5)

dl-o1.c (78 bytes) - added by jameszxj on Sep 29, 2019 at 8:13:58 AM.
dl-o2.c (78 bytes) - added by jameszxj on Sep 29, 2019 at 8:14:12 AM.
dl-load.c (1.7 KB) - added by jameszxj on Sep 29, 2019 at 8:14:30 AM.
0001-fix-rtems_rtl_elf_find_symbol-bug-when-use-dynamic-l.patch (1.2 KB) - added by jameszxj on Sep 29, 2019 at 8:35:12 AM.
patch
0001-dl01-Added-incrementally-linked-object-test-case.patch (9.2 KB) - added by Lou Woods on Dec 6, 2019 at 8:01:50 PM.
Added incrementally linked object test case to dl01 that works with rtems-tester

Download all attachments as: .zip

Change History (21)

comment:1 Changed on Apr 26, 2019 at 9:10:02 PM by Chris Johns

I have updated dl01 adding the example code and build PowerPC and ARM bsps. The PowerPC BSP works while the ARM BSP does not work.

The results for the psim BSP is:

$ rtems-run --rtems-bsp=psim `find . -name dl01.exe`
RTEMS Testing - Run, 5.0.not_released
 Command Line: /opt/work/rtems/5/bin/rtems-run --rtems-bsp=psim ./powerpc-rtems5/c/psim/testsuites/libtests/dl01.exe
 Python: 3.6.6 (default, Oct  2 2018, 01:22:29) [GCC 4.2.1 Compatible FreeBSD Clang 6.0.0 (tags/RELEASE_600/final 326565)]
Host: FreeBSD-12.0-RELEASE-p3-amd64-64bit-ELF (FreeBSD ruru 12.0-RELEASE-p3 FreeBSD 12.0-RELEASE-p3 GENERIC amd64 amd64)
OpenPIC Version ? (1 CPUs and 1 IRQ sources) at 0x202571776
OpenPIC Vendor 0 (Unknown), Device 0 (Unknown), Stepping 0
Overriding NumSources (1) from configuration with 16
OpenPIC timer frequency is not set
BATs must not overlap; area 0x08000000..0x09000000 hits DBAT 0
BATs must not overlap; area 0x0c000000..0x0d000000 hits DBAT 0
*** BEGIN OF TEST libdl (RTL) 1 ***
*** TEST VERSION: 5.0.0.be50969881b97180bf4fc1e2975efd41169e08bb-modified
*** TEST STATE: EXPECTED-PASS
*** TEST BUILD: RTEMS_POSIX_API
*** TEST TOOLS: 7.4.0 20181206 (RTEMS 5, RSB d9f49fc117422ea4b039cebc668191e9c07fde25, Newlib 1d35a003f)
load: /dl01-o1.o
handle: 0x6f668 loaded
Loaded module: argc:2 [/opt/work/chris/rtems/kernel/rtems.git/c/src/../../testsuites/libtests/dl01/dl-o1.c]
  0: Line 1
  1: Line 2
Loaded module: argc:3 [/opt/work/chris/rtems/kernel/rtems.git/c/src/../../testsuites/libtests/dl01/dl-o1.c]
  0: Call 2, line 1
  1: Call 2, line 2
  2: Call 2, line 3
handle: 0x6f668 closed
load: /dl01-o2o3.o
handle: 0x6f668 loaded
FOO Foo 1
BAR Bar 1
handle: 0x6f668 closed
*** END OF TEST libdl (RTL) 1 ***
*** FATAL ***
fatal source: 5 (RTEMS_FATAL_SOURCE_EXIT)
fatal code: 0 (0x00000000)
RTEMS version: 5.0.0.be50969881b97180bf4fc1e2975efd41169e08bb-modified
RTEMS tools: 7.4.0 20181206 (RTEMS 5, RSB d9f49fc117422ea4b039cebc668191e9c07fde25, Newlib 1d35a003f)
executing thread ID: 0x08a010001
executing thread name: UI1
Run time     : 0:00:01.032580

The results for the xilinx_zynq_a9_qemu is:

$ rtems-run --rtems-bsp=xilinx_zynq_a9_qemu `find . -name dl01.exe`
RTEMS Testing - Run, 5.0.not_released
 Command Line: /opt/work/rtems/5/bin/rtems-run --rtems-bsp=xilinx_zynq_a9_qemu ./arm-rtems5/c/xilinx_zynq_a9_qemu/testsuites/libtests/dl01.exe
 Python: 3.6.6 (default, Oct  2 2018, 01:22:29) [GCC 4.2.1 Compatible FreeBSD Clang 6.0.0 (tags/RELEASE_600/final 326565)]
Host: FreeBSD-12.0-RELEASE-p3-amd64-64bit-ELF (FreeBSD ruru 12.0-RELEASE-p3 FreeBSD 12.0-RELEASE-p3 GENERIC amd64 amd64)

(process:90678): GLib-WARNING **: 07:02:48.995: gmem.c:490: custom memory allocation vtable not supported
Warning: nic cadence_gem.0 has no peer
Warning: nic cadence_gem.1 has no peer


*** BEGIN OF TEST libdl (RTL) 1 ***
*** TEST VERSION: 5.0.0.be50969881b97180bf4fc1e2975efd41169e08bb-modified
*** TEST STATE: EXPECTED-PASS
*** TEST BUILD: RTEMS_POSIX_API
*** TEST TOOLS: 7.4.0 20181206 (RTEMS 5, RSB d9f49fc117422ea4b039cebc668191e9c07fde25, Newlib 1d35a003f)
load: /dl01-o1.o
handle: 0x117c98 loaded
Loaded module: argc:2 [/opt/work/chris/rtems/kernel/rtems.git/c/src/../../testsuites/libtests/dl01/dl-o1.c]
  0: Line 1
  1: Line 2
Loaded module: argc:3 [/opt/work/chris/rtems/kernel/rtems.git/c/src/../../testsuites/libtests/dl01/dl-o1.c]
  0: Call 2, line 1
  1: Call 2, line 2
  2: Call 2, line 3
handle: 0x117c98 closed
load: /dl01-o2o3.o
handle: 0x117c98 loaded
FOO Foo 1
FOO Foo 1
handle: 0x117c98 closed

*** END OF TEST libdl (RTL) 1 ***


*** FATAL ***
fatal source: 5 (RTEMS_FATAL_SOURCE_EXIT)
fatal code: 0 (0x00000000)
RTEMS version: 5.0.0.be50969881b97180bf4fc1e2975efd41169e08bb-modified
RTEMS tools: 7.4.0 20181206 (RTEMS 5, RSB d9f49fc117422ea4b039cebc668191e9c07fde25, Newlib 1d35a003f)
executing thread ID: 0x08a010001
executing thread name: UI1

Run time     : 0:00:01.018158

comment:2 Changed on Sep 25, 2019 at 6:51:23 PM by Joel Sherrill

Unless someone can point to a failure I am missing, I think this should be closed. I would appreciate Chris or Peter closing this if my testing below demonstrates this is no longer an issue.

Since Chris says this test case has been added to dl01. dl01 passes on xilinx_zynq_a9_qemu with results posted at https://lists.rtems.org/pipermail/build/2019-September/003639.html

For reference, this is the Zynq output which I think is correct:

*** BEGIN OF TEST libdl (RTL) 1 ***
*** TEST VERSION: 5.0.0.8798372261ed1df999bc9f4f3f0be0a230480041
*** TEST STATE: EXPECTED-PASS
*** TEST BUILD: RTEMS_POSIX_API
*** TEST TOOLS: 7.4.1 20190514 (RTEMS 5, RSB 83e0c3ddb8b3474806c9dad4efe4ad6afb0bb937, Newlib 6661a67)
load: /dl01-o1.o
handle: 0x1124b8 loaded
Loaded module: argc:2 [../../../../../../rtems/c/src/../../testsuites/libtests/dl01/dl-o1.c]
  0: Line 1
  1: Line 2
Loaded module: argc:3 [../../../../../../rtems/c/src/../../testsuites/libtests/dl01/dl-o1.c]
  0: Call 2, line 1
  1: Call 2, line 2
  2: Call 2, line 3
handle: 0x1124b8 closed

*** END OF TEST libdl (RTL) 1 ***


*** FATAL ***
fatal source: 5 (RTEMS_FATAL_SOURCE_EXIT)
fatal code: 0 (0x00000000)
RTEMS version: 5.0.0.8798372261ed1df999bc9f4f3f0be0a230480041
RTEMS tools: 7.4.1 20190514 (RTEMS 5, RSB 83e0c3ddb8b3474806c9dad4efe4ad6afb0bb937, Newlib 6661a67)
executing thread ID: 0x08a010001
executing thread name: UI1 

Can this be closed?

comment:3 Changed on Sep 29, 2019 at 2:02:40 AM by jameszxj

no, it still has a failure, not libtests/dl01, but the test example in the mail
https://lists.rtems.org/pipermail/devel/2019-April/025623.html

I think I have found the bug, but no one notice the mail I ever wrote. Would you please have a look at the mail https://lists.rtems.org/pipermail/devel/2019-September/055590.html

comment:4 in reply to:  3 Changed on Sep 29, 2019 at 5:13:32 AM by Chris Johns

Replying to jameszxj:

no, it still has a failure, not libtests/dl01, but the test example in the mail
https://lists.rtems.org/pipermail/devel/2019-April/025623.html

I think I have found the bug, but no one notice the mail I ever wrote. Would you please have a look at the mail https://lists.rtems.org/pipermail/devel/2019-September/055590.html

Thank you for updating the ticket. If you could create a patch for the code and the test case and attach it here it would be a great help.

Changed on Sep 29, 2019 at 8:13:58 AM by jameszxj

Attachment: dl-o1.c added

Changed on Sep 29, 2019 at 8:14:12 AM by jameszxj

Attachment: dl-o2.c added

Changed on Sep 29, 2019 at 8:14:30 AM by jameszxj

Attachment: dl-load.c added

comment:5 Changed on Sep 29, 2019 at 8:19:52 AM by jameszxj

I complite test case use the follow command

arm-rtems5-gcc  -march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard -mtune=cortex-a9 -mlong-calls -Wall -I. -O2 -c -o wsp/build/dl-o2.o src/dl-o2.c
arm-rtems5-gcc  -march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard -mtune=cortex-a9 -mlong-calls -Wall -I. -O2 -c -o wsp/build/dl-o1.o src/dl-o1.c
arm-rtems5-ld -r -o wsp/build/dl.out wsp/build/dl-o2.o wsp/build/dl-o1.o

output message:

handle: 0x2207620 loaded
dl_o1_func
dl_o1_func
handle: 0x2207620 closed

The right output should be:

handle: 0x2207620 loaded
dl_o1_func
dl_o2_func
handle: 0x2207620 closed

comment:6 Changed on Sep 29, 2019 at 8:33:08 AM by jameszxj

I change the condition from STT_NOTYPE to STB_GLOBAL. It runs OK on my z7000 board, and I try it in my project(origial use RAP foramt),
it works too.

comment:7 Changed on Sep 29, 2019 at 10:16:59 PM by Chris Johns

Thank you for the patch. I am not sure we can remove the check for STT_NOTYPE so should the change be ...

if (ELF_ST_BIND(sym->st_info) == STB_GLOBAL || 
    ELF_ST_TYPE(sym->st_info) == STT_NOTYPE || 
    sym->st_shndx == SHN_COMMON)
{
 ...

I am concerned removing STT_NOTYPE may break other architectures.

comment:8 Changed on Oct 8, 2019 at 1:57:11 AM by jameszxj

This does not work on arm. From IHI0044E_aaelf.pdf section 4.5.2 , STT_NOTYPE does not mean a global symbol.

The function if(...) is to make a distinction between GLOBAL symbols and LOCAL symbols, so use (ELF_ST_TYPE(sym->st_info) == STT_NOTYPE ) is wrong on ARM architectures.
BTW: (ELF_ST_TYPE(sym->st_info) == STT_FUNC) works too on ARM.

But your concern is reasonable, I think we should test on diffrent architectures.

comment:9 Changed on Nov 20, 2019 at 4:22:21 PM by Lou Woods

I've successfully run jameszxj's patch and example code on these architectures/BSPs: Arm/xilinx_zynq_a9_qemu, PowerPC/psim, SPARC/erc32, and i386/pc386. I also ran tests DL01-09 on each stated BSP with and without the fix and ensured that the fix didn’t break any of these tests.

comment:10 in reply to:  9 ; Changed on Nov 20, 2019 at 10:46:49 PM by Chris Johns

Replying to Lou Woods:

I've successfully run jameszxj's patch and example code on these architectures/BSPs: Arm/xilinx_zynq_a9_qemu, PowerPC/psim, SPARC/erc32, and i386/pc386. I also ran tests DL01-09 on each stated BSP with and without the fix and ensured that the fix didn’t break any of these tests.

Thank you. I assume dl06 and dl09 failed as expected?

comment:11 in reply to:  10 ; Changed on Nov 21, 2019 at 3:29:47 PM by Lou Woods

Replying to Chris Johns:

Replying to Lou Woods:

I've successfully run jameszxj's patch and example code on these architectures/BSPs: Arm/xilinx_zynq_a9_qemu, PowerPC/psim, SPARC/erc32, and i386/pc386. I also ran tests DL01-09 on each stated BSP with and without the fix and ensured that the fix didn’t break any of these tests.

Thank you. I assume dl06 and dl09 failed as expected?

dl06 failed on all. dl09 only failed on erc32, also dl05 failed on pc386.

comment:12 in reply to:  11 ; Changed on Nov 21, 2019 at 7:31:42 PM by Chris Johns

Replying to Lou Woods:

dl06 failed on all. dl09 only failed on erc32, also dl05 failed on pc386.

Thanks.

Was the result the same without the patch? I suspect it was, as we do not have a test that incrementally links object files. I think it would be good to have one.

I am fine with this patch being push on condition a test is added. If you cannot add a test, that is fine, however I would prefer the patch remains here until we have one.

A complication is this is a change to the build system and would effect Sebastian. I would like his input as well.

comment:13 in reply to:  12 ; Changed on Nov 21, 2019 at 10:20:55 PM by Lou Woods

Replying to Chris Johns:

Replying to Lou Woods:

dl06 failed on all. dl09 only failed on erc32, also dl05 failed on pc386.

Thanks.

Was the result the same without the patch?

Yes

I suspect it was, as we do not have a test that incrementally links object files. I think it would be good to have one.

I am fine with this patch being push on condition a test is added. If you cannot add a test, that is fine, however I would prefer the patch remains here until we have one.

I don't think it will be too hard to convert what jameszxj provided into an automated test. I don't think it is a problem to do that.

A complication is this is a change to the build system and would effect Sebastian. I would like his input as well.

I can hold off until Sebastian has weighed in as well.

comment:14 in reply to:  13 Changed on Nov 21, 2019 at 11:15:49 PM by Chris Johns

Replying to Lou Woods:

Replying to Chris Johns:

Replying to Lou Woods:

dl06 failed on all. dl09 only failed on erc32, also dl05 failed on pc386.

Thanks.

Was the result the same without the patch?

Yes

OK this means we need a test.

I suspect it was, as we do not have a test that incrementally links object files. I think it would be good to have one.

I am fine with this patch being push on condition a test is added. If you cannot add a test, that is fine, however I would prefer the patch remains here until we have one.

I don't think it will be too hard to convert what jameszxj provided into an automated test. I don't think it is a problem to do that.

Thanks. I would add it to dl01 unless you think it is best in another test. I do not see a need for a new test.

A complication is this is a change to the build system and would effect Sebastian. I would like his input as well.

I can hold off until Sebastian has weighed in as well.

Great, we need to know if we can change the current build system.

Changed on Dec 6, 2019 at 8:01:50 PM by Lou Woods

Added incrementally linked object test case to dl01 that works with rtems-tester

comment:15 Changed on Dec 6, 2019 at 8:10:08 PM by Lou Woods

The above test case patch uses the current build system. Once the new build system is settled it can be converted and hopefully merged along with the fix.

comment:16 Changed on Feb 26, 2020 at 3:53:43 AM by Chris Johns

Milestone: 5.16.1
Note: See TracTickets for help on using tickets.