Long path crashes the RSB when listing a directory.

[Chris Johns]

Building LM32 on Windows crashes the RSB with a long path. The os.listdir call in Python on Windows is limited to 254 characters even if the path is Uncode.

building: lm32-rtems4.11-gcc-4.9.3-newlib-2.2.0.20150423-x86_64-w64-mingw32-1
Build Set: Time 0:29:19.809228
Build Set: Time 3:47:43.385503
Traceback (most recent call last):
  File "../source-builder/sb-set-builder", line 29, in <module>
    setbuilder.run()
  File "../source-builder/sb/setbuilder.py", line 502, in run
    b.build(deps)
  File "../source-builder/sb/setbuilder.py", line 340, in build
    bs.build(deps, nesting_count)
  File "../source-builder/sb/setbuilder.py", line 354, in build
    self.build_package(configs[s], b)
  File "../source-builder/sb/setbuilder.py", line 194, in build_package
    _build.config.expand('%{_tmproot}'))
  File "../source-builder/sb/setbuilder.py", line 155, in root_copy
    self.copy(src, dst)
  File "../source-builder/sb/setbuilder.py", line 95, in copy
    path.copy_tree(src, dst)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 232, in copy_tree
    copy_tree(srcname, dstname)
  File "../source-builder/sb/path.py", line 191, in copy_tree
    names = os.listdir(hsrc)
TypeError: encoded string too long (269, maximum length 259)

[Chris Johns]

I have looked into this issue in detail and it is complicated. At the heart of the problem is the inherent path limit in the WIN32 API which is documented in ​Naming Files, Paths, and Namespaces. There is also ​Python Issue 27731 which details how Python will respond.

I see a couple of solution, one complicated and full of potential issues which others have solved, for example the MSYS2 shell, and the second is a simple hack. The complex solution is to step down to the Win32 API and selectively use Win32 API calls that support the Unicode extension to let the RSB work. I feel this is a lot of work and difficult to get to work. The second solution is to generate a script and to run it and let the shell on the host manage the issue. The RSB needs a shell to build most packages so it has to be present.

We see the long file name issue when coping a tree and removing a tree.

[Chris Johns]

Fixed with the patch 78c1524/rtems-source-builder.

[Chris Johns]

This 4.11 fix needs to be ported to 4.12.

[Sebastian Huber]

Milestone renamed

[Chris Johns]

The patch in this ticket is applied to master.