tclsh required to build sqlite -- makes all BSP bsets fail

[Joel Sherrill]

I'm not sure what to do about this. If building sqlite is a requirement, then host setup instructions, sb-check, etc are impacted. But dtc is a required component for a few things and the RSB deals with it. It is also possible to say that sqlite should not be in the BSP bsets.

This is IMO a blocker at some level for the 5.1 release because it impacts building all bsets.

+ make -j 8 sqlite3.h libsqlite3.la
tclsh /home/joel/rtems-cron-5/rtems-source-builder/rtems/build/sqlite-3.8.8.1-powerpc-rtems5-1/build-xc/../sqlite-src-3080801/tool/mksqlite3h.tcl /home/joel/rtems-cron-5/rtems-source-builder/rtems/build/sqlite-3.8.8.1-powerpc-rtems5-1/build-xc/../sqlite-src-3080801 >sqlite3.h
gcc  -g -o mkkeywordhash -DSQLITE_OMIT_WAL=1 -DSQLITE_ENABLE_COLUMN_METADATA=1  /home/joel/rtems-cron-5/rtems-source-builder/rtems/build/sqlite-3.8.8.1-powerpc-rtems5-1/build-xc/../sqlite-src-3080801/tool/mkkeywordhash.c
gcc  -g -o lemon /home/joel/rtems-cron-5/rtems-source-builder/rtems/build/sqlite-3.8.8.1-powerpc-rtems5-1/build-xc/../sqlite-src-3080801/tool/lemon.c
/bin/sh: tclsh: command not found

[Joel Sherrill]

Advice from Jonathan Brandmeyer (​https://lists.rtems.org/pipermail/devel/2020-April/058905.html).

---

We just copied the amalgamation's single .c and .h file into our repository and built it as a single object into our application. So the following recommendations come only from an amateur reading of the sources on current rtems-source-builder master.

3.8.8 is pretty old. I'd just jump straight to 3.30.1, the current release.

Instead of downloading the sqlite-src zip, download sqlite-amalgamation or sqlite-autoconf and use that as a base. I think the bset's configuration invocation is OK, except that I would set different CFLAGS as described in the next couple of paragraphs. See also ​https://sqlite.org/amalgamation.html

By default, the SQLite write-ahead-log relies on mmap to share some of its index structures between multiple processes. It looks like the current RTEMS bset disables the WAL entirely. Using the WAL gives much better performance than the rollback journal for write-intensive use cases. We used SQLite to reliably buffer up and stitch together segments of our field software update process, which fits the WAL very well. We set -DSQLITE_MAX_MMAP_SIZE=0 -DSQLITE_DEFAULT_LOCKING_MODE=1 at compile time to avoid mmap's use since RTEMS has neither a VMM nor multiple address spaces. It is still up to the application to choose either the WAL or the rollback journal. See also ​https://sqlite.org/wal.html#noshm

We also added -DSQLITE_ENABLE_MEMSYS5 to our build, and used it to give SQLite its own memory spaces to operate on distinct from the rest of our application. We explicitly provided it with its own MEMSYS5 heap, page cache, and lookaside pools. RTEMS just needs to add the correct compile-time option to support this functionality. It is still up to the application to configure and monitor those memory pools at runtime. See also ​https://sqlite.org/malloc.html

---

[Jonathan Brandmeyer <jbrandmeyer@…>]

In 113c65c/rtems-source-builder:

databases/sqlite: Update to 3.31.1

closes #3927.