#3132 closed enhancement (fixed)

Add reference counting to file descriptors

Reported by: Sebastian Huber Owned by: Sebastian Huber
Priority: normal Milestone: 5.1
Component: fs Version: 5
Severity: normal Keywords:
Cc: Blocked By:
Blocking:

Description (last modified by Sebastian Huber)

The use of a file descriptor after or during a close() operation may result in a use after free. Finding such errors in applications is difficult. Especially in SMP systems using the highly dynamic libbsd network stack.

The file descriptor objects reside in a table with a application configuration defined size. So, the storage of a file descriptor object is always present, only the referenced file system node may change over time. The file system nodes may use an internal reference counting, which is independent of the file descriptors.

To implement reference counting for the file descriptors add a bit field for the reference count to the rtems_libio_t::flags and use atomic operations to maintain the flags.

Each operation using a file descriptor should perform a sequence like this:

int op( int fd, ... )
{
  rtems_libio_t *iop;
  unsigned int   flags;

  if ( (uint32_t) fd >= rtems_libio_number_iops ) {
    rtems_set_errno_and_return_minus_one( EBADF );
  }

  iop = rtems_libio_iop( fd );
  flags = rtems_libio_iop_hold( iop );

  if ( ( flags & LIBIO_FLAGS_OPEN ) == 0 ) {
    rtems_libio_iop_drop( _iop );
    rtems_set_errno_and_return_minus_one( EBADF );
  }

  do_op( iop, ... );
  rtems_libio_iop_drop( iop );
  return 0;
}

A close() should return -1 with EBUSY in case the file descriptor is referenced. In this case, no close operation will be performed.

Change History (22)

comment:1 Changed on Sep 13, 2017 at 5:56:49 AM by Sebastian Huber

Status: assignedaccepted

comment:2 Changed on Sep 14, 2017 at 5:03:35 AM by Sebastian Huber <sebastian.huber@…>

In ead1281f/rtems:

bsp/mrm332: Remove dead code

Update #3132.

comment:3 Changed on Sep 14, 2017 at 6:36:11 AM by Sebastian Huber

Description: modified (diff)

comment:4 Changed on Sep 15, 2017 at 8:58:46 AM by Sebastian Huber <sebastian.huber@…>

In 7b45202/rtems:

libio: Simplify rtems_libio_iop()

Remove the file descriptor validation. This is the job of
rtems_libio_check_fd(). Use an inline function instread of a macro.

Update #3132.

comment:5 Changed on Sep 15, 2017 at 8:58:57 AM by Sebastian Huber <sebastian.huber@…>

In 4b759b1/rtems:

libio: Avoid direct use of rtems_libio_iops

Update #3132.

comment:6 Changed on Sep 15, 2017 at 8:59:09 AM by Sebastian Huber <sebastian.huber@…>

In 0169e90/rtems:

libio: Do simple parameter checks early

This simplifies error handling later.

Update #3132.

comment:7 Changed on Sep 15, 2017 at 8:59:20 AM by Sebastian Huber <sebastian.huber@…>

In 48dbb6cf/rtems:

libio: Remove rtems_libio_check_permissions()

Remove rtems_libio_check_permissions() and convert single user to
rtems_libio_check_permissions_with_error().

Update #3132.

comment:8 Changed on Sep 15, 2017 at 8:59:32 AM by Sebastian Huber <sebastian.huber@…>

In ec10d26/rtems:

libio: rtems_libio_check_permissions_with_error()

Rename rtems_libio_check_permissions_with_error() in
rtems_libio_check_permissions().

Update #3132.

comment:9 Changed on Sep 15, 2017 at 8:59:43 AM by Sebastian Huber <sebastian.huber@…>

In 856ede4f/rtems:

libio: Add iop set/clear flags

Update #3132.

comment:10 Changed on Sep 15, 2017 at 8:59:56 AM by Sebastian Huber <sebastian.huber@…>

In ca90c6c/rtems:

libio: Add rtems_libio_iop_flags_initialize()

Update #3132.

comment:11 Changed on Sep 15, 2017 at 9:00:08 AM by Sebastian Huber <sebastian.huber@…>

In e2b1db23/rtems:

libio: Add rtems_libio_iop_flags()

Update #3132.

comment:12 Changed on Sep 15, 2017 at 9:00:20 AM by Sebastian Huber <sebastian.huber@…>

In bbcdc302/rtems:

libio: Add rtems_libio_iop_is_no_delay()

Update #3132.

comment:13 Changed on Sep 15, 2017 at 9:00:32 AM by Sebastian Huber <sebastian.huber@…>

In a937a5a/rtems:

libio: Add rtems_libio_iop_is_readable()

Update #3132.

comment:14 Changed on Sep 15, 2017 at 9:00:45 AM by Sebastian Huber <sebastian.huber@…>

In 3cffd66d/rtems:

libio: Add rtems_libio_iop_is_writeable()

Update #3132.

comment:15 Changed on Sep 15, 2017 at 9:00:57 AM by Sebastian Huber <sebastian.huber@…>

In d4c5441/rtems:

libio: Add rtems_libio_iop_is_append()

Update #3132.

comment:16 Changed on Sep 15, 2017 at 9:01:09 AM by Sebastian Huber <sebastian.huber@…>

In 9012db8/rtems:

libio: LIBIO_GET_IOP() LIBIO_GET_IOP_WITH_ACCESS()

Replace rtems_libio_check_fd(), rtems_libio_iop(),
rtems_libio_check_open() and rtems_libio_check_permissions()
combinations with new LIBIO_GET_IOP() and LIBIO_GET_IOP_WITH_ACCESS()
macros.

Update #3132.

comment:17 Changed on Sep 15, 2017 at 9:01:20 AM by Sebastian Huber <sebastian.huber@…>

In 98041b68/rtems:

libio: Unify readv() and writev()

Update #3132.

comment:18 Changed on Sep 15, 2017 at 9:01:32 AM by Sebastian Huber <sebastian.huber@…>

In baef823c/rtems:

libio: Add hold/drop iop reference

Check iop reference count in close() and return -1 with errno set to
EBUSY in case the file descriptor is still in use.

Update #3132.

comment:19 Changed on Sep 15, 2017 at 10:49:57 AM by Sebastian Huber <sebastian.huber@…>

Resolution: fixed
Status: acceptedclosed

In 894c965/rtems-libbsd:

Support reference counting for file descriptors

Close #3132.

comment:20 Changed on Nov 9, 2017 at 6:27:14 AM by Sebastian Huber

Milestone: 4.12.05.1

Milestone renamed

comment:21 Changed on Nov 15, 2017 at 12:23:13 PM by Sebastian Huber <sebastian.huber@…>

In d4b99ae/rtems:

libio: Add assert to rtems_libio_iop_drop()

This assert helps to detect an invalid reference counting in RTEMS_DEBUG
configurations.

Update #3132.

comment:22 Changed on Nov 15, 2017 at 12:25:56 PM by Sebastian Huber <sebastian.huber@…>

In b03a1c0/rtems-libbsd:

Fix file descriptor reference counting in accept()

Update #3132.

Note: See TracTickets for help on using tickets.