Opened on 06/07/18 at 07:30:17
Last modified on 12/17/21 at 16:16:53
#3448 assigned task
Add GCC, Binutils+GDB, Newlib mirrors to RTEMS github
Reported by: | Sebastian Huber | Owned by: | Sebastian Huber |
---|---|---|---|
Priority: | normal | Milestone: | Indefinite |
Component: | tool | Version: | 5 |
Severity: | normal | Keywords: | |
Cc: | Blocked By: | ||
Blocking: |
Description
Sometimes it is necessary to use a snapshot of GCC, Binutils+GDB or Newlib. The snapshot archives of GCC and Binutils+GDB are only available for a certain time (e.g. half a year).
https://lists.rtems.org/pipermail/devel/2018-June/021907.html
The sourceware.org Git services do not provide snapshots.
One option would be to use github since it provides snapshots. However, no official mirrors at github exist.
Add mirrors for GCC, Binutils+GDB and Newlib to https://github.com/rtems.
Change History (17)
comment:1 Changed on 06/11/18 at 08:31:19 by Sebastian Huber
Owner: | changed from joel to Sebastian Huber |
---|
comment:2 Changed on 06/11/18 at 09:29:49 by Chris Johns
What is practical?
The servers are on a fast backbone and so I do not see a big issue here.
comment:3 Changed on 06/11/18 at 09:51:22 by Sebastian Huber
The least maintenance overhead has probably an automated mirroring of all branches since there are standard Git commands to do this.
comment:4 Changed on 06/11/18 at 09:56:25 by Sebastian Huber
How are is the Git mirroring done for the RTEMS repositories? Can we use the same Git user account for this?
comment:5 Changed on 06/11/18 at 14:17:40 by Gedare Bloom
This account https://github.com/rtems-updater
is used from dispatch to push commits in a git hook.
I don't know who has the rtems-updater credentials,
probably Amar set it up. I would guess its private key
should be on dispatch though.
comment:6 Changed on 06/12/18 at 01:48:22 by Chris Johns
Can the repos please be under something like 'misc' or 'gnu' or 'fsf' and not at the top level.
I am concerned being at the top level will imply this is what users need to use.
Thanks
comment:7 follow-up: 9 Changed on 06/12/18 at 05:35:32 by Sebastian Huber
I added the repositories.
I don't know you can add subdirectories in Github. Is this possible at all?
We could use some sort of prefix, e.g. mirror-gcc, mirror-binutils-gdb, mirror-newlib-cygwin.
comment:8 follow-up: 10 Changed on 06/12/18 at 05:37:34 by Sebastian Huber
The git clone --mirror checkout of the three repository need 2.5G of space on my disk. On dispatch.rtems.org we have currently:
df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/gpt/root0 88G 80G 1.4G 98% /
comment:9 follow-up: 11 Changed on 06/12/18 at 07:07:06 by Chris Johns
Replying to Sebastian Huber:
I don't know you can add subdirectories in Github. Is this possible at all?
I thought you could but I must have not understood what I was looking at. I see there are pinned repos which might help us if we pin our ones so they are at the top.
We could use some sort of prefix, e.g. mirror-gcc, mirror-binutils-gdb, mirror-newlib-cygwin.
That is a good idea. fsf-mirror-gcc?.
comment:10 Changed on 06/12/18 at 07:08:08 by Chris Johns
Replying to Sebastian Huber:
The git clone --mirror checkout of the three repository need 2.5G of space on my disk. On dispatch.rtems.org we have currently:
df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/gpt/root0 88G 80G 1.4G 98% /
What if we use the NAS under /data/ftp?
comment:11 follow-up: 12 Changed on 06/12/18 at 07:57:08 by Sebastian Huber
Replying to Chris Johns:
Replying to Sebastian Huber:
I don't know you can add subdirectories in Github. Is this possible at all?
I thought you could but I must have not understood what I was looking at. I see there are pinned repos which might help us if we pin our ones so they are at the top.
I added pinned repositories.
We could use some sort of prefix, e.g. mirror-gcc, mirror-binutils-gdb, mirror-newlib-cygwin.
That is a good idea. fsf-mirror-gcc?.
What about gnu-mirror-gcc, sourceware-mirror-newlib-cygwin, and sourceware-mirror-binutils-gdb? This reflects the URL of the upstream Git repositories somehow.
comment:12 Changed on 06/12/18 at 22:33:51 by Chris Johns
Replying to Sebastian Huber:
Replying to Chris Johns:
Replying to Sebastian Huber:
I don't know you can add subdirectories in Github. Is this possible at all?
I thought you could but I must have not understood what I was looking at. I see there are pinned repos which might help us if we pin our ones so they are at the top.
I added pinned repositories.
Thanks. Looks good.
I am wondering if I change the "Kernel, networking, file-systems, drivers, BSPs, samples, and testsuite." description on rtems to "Kernel (SMP), file-systems, drivers, BSPs, samples, and testsuite." and libbsd to "RTEMS FreeBSD, networking, USB, and media." ?
We could use some sort of prefix, e.g. mirror-gcc, mirror-binutils-gdb, mirror-newlib-cygwin.
That is a good idea. fsf-mirror-gcc?.
What about gnu-mirror-gcc, sourceware-mirror-newlib-cygwin, and sourceware-mirror-binutils-gdb? This reflects the URL of the upstream Git repositories somehow.
Nice. That makes sense to me.
comment:13 follow-up: 14 Changed on 06/13/18 at 05:33:44 by Sebastian Huber
I have now the following setup on a local machine:
git clone --mirror git://sourceware.org/git/binutils-gdb.git cd binutils-gdb git remote add github git@github.com:RTEMS/sourceware-mirror-binutils-gdb.git
git clone --mirror git://sourceware.org/git/newlib-cygwin.git cd newlib-cygwin git remote add github git@github.com:RTEMS/sourceware-mirror-newlib-cygwin.git
git clone --mirror git://gcc.gnu.org/git/gcc.git/git/binutils-gdb.git cd gcc git remote add github git@github.com:RTEMS/gnu-mirror-gcc.git
Synchronization is quite easy:
git fetch git push --mirror github
I will set up a cron job on my local machine for now. It runs during work days. To set it up on dispatch.rtems.org we need more disk space (I suggest 4G) and someone with enough privileges on this server.
comment:14 Changed on 06/13/18 at 05:58:49 by Chris Johns
Replying to Sebastian Huber:
I will set up a cron job on my local machine for now. It runs during work days. To set it up on dispatch.rtems.org we need more disk space (I suggest 4G) and someone with enough privileges on this server.
Your local ftp
directory under your personal area is on the NAS and should have enough space. Note, the NAS's FTP may not appear in the du
list.
You can chmod
the work space to stop it being scanned by search bots.
comment:15 Changed on 06/18/18 at 06:38:44 by Sebastian Huber
I have at least two problems to set this up on dispatch.rtems.org:
- I cannot add a user cron job:
dispatch:~ # crontab -u sebh -e crontab: must be privileged to use -u dispatch:~ # sudo crontab -u sebh -e -bash: sudo: command not found
- The Git mirror script forwards the request to a local server on port 8003 (/data/support/git-support/hooks/github-cmd-send.py). This server probably needs to have access to the local Git clone. I cannot set this up.
comment:16 Changed on 12/19/19 at 08:15:22 by Sebastian Huber
Milestone: | 5.1 → Indefinite |
---|
comment:17 Changed on 12/17/21 at 16:16:53 by Joel Sherrill
Doesn't this work now? Sebastian/Chris? would you close the ticket or clarify what remains to do?
My first step is to add
repositories, starting with the master branches only.
The second step is to add a cron job to dispatch.rtems.org to synchronize the repositories once per day with the upstream repositories.
Do we want to mirror all branches of the upstream repositories?