#2992 closed defect (fixed)

Long path crashes the RSB when listing a directory.

Reported by: Chris Johns Owned by: Chris Johns
Priority: normal Milestone: 5.1
Component: tool/rsb Version: 5
Severity: major Keywords: RSB
Cc: Blocked By:
Blocking:

Description

Building LM32 on Windows crashes the RSB with a long path. The os.listdir call in Python on Windows is limited to 254 characters even if the path is Uncode.

building: lm32-rtems4.11-gcc-4.9.3-newlib-2.2.0.20150423-x86_64-w64-mingw32-1
Build Set: Time 0:29:19.809228
Build Set: Time 3:47:43.385503
Traceback (most recent call last):
  File "../source-builder/sb-set-builder", line 29, in <module>
    setbuilder.run()
  File "../source-builder/sb/setbuilder.py", line 502, in run
    b.build(deps)
  File "../source-builder/sb/setbuilder.py", line 340, in build
    bs.build(deps, nesting_count)
  File "../source-builder/sb/setbuilder.py", line 354, in build
    self.build_package(configs[s], b)
  File "../source-builder/sb/setbuilder.py", line 194, in build_package
    _build.config.expand('%{_tmproot}'))
  File "../source-builder/sb/setbuilder.py", line 155, in root_copy
    self.copy(src, dst)
  File "../source-builder/sb/setbuilder.py", line 95, in copy
    path.copy_tree(src, dst)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 191, in copy_tree
    names = os.listdir(hsrc)
TypeError: encoded string too long (269, maximum length 259)

Change History (5)

comment:1 Changed on Apr 18, 2017 at 12:34:38 AM by Chris Johns

I have looked into this issue in detail and it is complicated. At the heart of the problem is the inherent path limit in the WIN32 API which is documented in Naming Files, Paths, and Namespaces. There is also Python Issue 27731 which details how Python will respond.

I see a couple of solution, one complicated and full of potential issues which others have solved, for example the MSYS2 shell, and the second is a simple hack. The complex solution is to step down to the Win32 API and selectively use Win32 API calls that support the Unicode extension to let the RSB work. I feel this is a lot of work and difficult to get to work. The second solution is to generate a script and to run it and let the shell on the host manage the issue. The RSB needs a shell to build most packages so it has to be present.

We see the long file name issue when coping a tree and removing a tree.

comment:2 Changed on Jun 15, 2017 at 2:46:32 AM by Chris Johns

Resolution: fixed
Status: assignedclosed

Fixed with the patch 78c1524/rtems-source-builder.

comment:3 Changed on Aug 7, 2017 at 2:16:18 AM by Chris Johns

Milestone: 4.11.24.12.0
Resolution: fixed
Status: closedreopened
Version: 4.114.12

This 4.11 fix needs to be ported to 4.12.

comment:4 Changed on Nov 9, 2017 at 6:27:14 AM by Sebastian Huber

Milestone: 4.12.05.1

Milestone renamed

comment:5 Changed on Oct 14, 2018 at 8:45:22 PM by Chris Johns

Resolution: fixed
Status: reopenedclosed

The patch in this ticket is applied to master.

Note: See TracTickets for help on using tickets.