wiki:Developer/CodeSwarm

Version 11 (modified by Chris Johns, on 11/22/14 at 04:57:34) (diff)

--

RTEMS Code Swarm

Code Swarm is a tool which visualizes project commit history. It can produce some very cool and interesting output.

Thanks to Peter Burns (a.k.a rictic) for fixing bugs and providing guidance during this. He was most supportive.

Running Code Swarm

# generate the RTEMS activity log from a checked out tree
cvs log >activity.log

# download and extract
curl -L http://github.com/rictic/code_swarm/tarball/master > code_swarm.tar.gz
tar -xzf code_swarm.tar.gz
mv rictic-code_swarm-*/ code_swarm

# add the bin directory to your path
PATH=$PATH:code_swarm/bin

# convert the CVS log 
convert_logs.py -c activity.log > log.xml
#get a code_swarm config file (I'd eventually like to remove this step)
cp code_swarm/bin/config.template rtems.config
#run code_swarm on the config file
code_swarm -c rtems.config

At this point, you will have a subdirectory in the code_swarm tree named code_swarm_frame with a lot of .png files. Following the instructions for Generating a Video, I used this command to generate the movie.avi file.

mencoder mf://*.png -mf fps=24:type=png -ovc lavc -oac copy -o movie.avi

If that results in a file that is too large, read the Wiki and try the other alternatives.

This takes between 90 minutes and 6 hours on the RTEMS activity log depending on how many frames per day you generate. At 2 frames per day, it is about 90 minutes and at 6, it is ~6 hours.

Open Issues

RTEMS has a history of having many submitters and few committers. Code Swarm uses the committer user Id. It would be nice if it could make a distinction between author and submitter and show the extra level of activity. Since from 1995 to 2002, JoelSherrill committed nearly all submissions, this looks very bad in the RTEMS video. During this period, to identify just one submitter, Eric Norum submitted the TCP/IP stack, termios, a handful of BSPs and other code. This is simply not reflected.

Bursts

Date(day/month/year): from 12/05/1995 - to 14/12/2011.

Video length: 8 minutes and 25 seconds.

Frames per second(fps):24.

Names floating in the burst Date(day:month name:year) Time in video(minute:second:frame)
joel12:May:199500:00:00
joel31:May:199500:01:15
joel12:September:199500:10:06
joel20:December:199500:18:12
joel22:April:199700:59:07
joel03:June:199701:02:18
joel08:October:199701:13:09
joel18:February:199801:24:10
joel, wade20:August:199801:39:16
joel, jennifer18:February:199901:54:22
joel, jennifer27:July:199902:08:02
joel, jennifer18:November:199902:17:14
joel, jennifer31:January:200002:23:19
joel, jennifer13:April:200002:29:21
joel12:June:200002:34:21
joel05:September:200002:41:23
joel, jennifer24:January:200102:53:18
joel18:January:200203:23:14
joel28:March:200203:29:08
joel, ralf11:August:200203:40:17
joel, jennifer, ralf, ccj, ericn11:February:200303:56:01
joel, jennifer, ralf, ccj16:August:200304:11:13
joel, jennifer, ralf, ccj05:September:200304:13:04
joel, jennifer, ralf14:January:200404:24:03
joel, jennifer, ralf, ccj31:March:200404:30:12
joel, ralf, ccj31:April:200404:32:07
joel, ralf, ccj, ericn, richard, loki, jtm, thomas06:October:200505:16:16
joel, ralf, ericn, richard, loki, jtm, thomas, strauman07:November:200505:19:09
joel, ralf, ericn, strauman12:January:200605:24:21
joel, ralf, ericn, strauman, ccj, thomas17:October:200605:48:01
joel, ralf, strauman, ccj, thomas02:December:200605:51:21
joel, ralf, strauman, ccj, thomas, ericn18:January:200705:55:19
joel, ralf, strauman, ccj, thomas, ericn12:March:200706:00:05
joel, ralf, strauman, ccj, thomas, humph11:December:200706:23:01
joel, ralf, strauman, ccj, thomas, humph, jennifer, ericn17:April:200806:33:16
joel, ralf, strauman, ccj, thomas, ericn17:September:200806:46:10
joel, ralf, strauman, ccj, thomas, ericn, jennifer15:October:200907:19:05
joel, ralf, strauman, ccj, thomas, ericn29:November:200907:22:23
joel, ralf, strauman, ccj, thomas, ericn, sh, humph30:April:201007:35:16
joel, ralf, strauman, ccj, thomas, ericn, sh, humph07:June:201007:38:19
joel, ralf, strauman, ccj, sh02:February:201107:58:19
joel, ralf, strauman, ccj, sh22:February:201108:00:11
joel, ralf, strauman, ccj, sh, jennifer25:July:201108:13:04
joel, ralf, strauman, ccj, sh, jennifer, gedare09:December:201108:24:14

Text for Video

The video shows periods of high and low activity. On the videos from other projects, there are subtitles indicating what actually happened in the project at that time.

  • beginning until non-OAR committers added - 2002/06/14 06:50:12 Ralf.
  • May 1995
  • December 1995
  • Around April 1997
  • June 1997
  • October 1997
  • Feb 1998
  • July 1999
  • November 1999
  • April 2000
  • June 2000
  • Jan 2002
  • April 2002
  • Feb 2003
  • August 2003
  • April 2004
  • October 2005
  • Jan 2007
  • September 2008

There has been a surge in the code commits in the above mentioned months.

Committers and their first commit:

  • joel, Joel Sherrill - 11 May 1995
  • jmj, Mark Johannes - 24 May 1996
  • wade, ? - 10 August 1998
  • jennifer, Jennifer Averrett, 2 September 1998
  • ralf, Ralf Corsepius - 16 June 2002
  • ericn, Eric Norum - 13 September 2002
  • ccj, Chris Johns - 31 October 2002
  • jtm, Jay Monkman - 17 July 2004
  • gregmenke, Greg Menke, 6 December 2004
  • strauman, Till Straumann - 12 January 2006
  • loki, Philippe Simons - 7 July 2005
  • richard, Richard Campbell - 16 September 2005
  • thomas, Thomas Doerfler - 29 September 2005
  • humph, Glenn Humphrey - 10 October 2007

RTEMS Configuration File

You can try enabling OpenGL support and see if it works for you.

# This is a sample configuration file for code_swarm

# Input file
InputFile=log.xml

# Optional Method instead of MillisecondsPerFrame
FramesPerDay=2

# OpenGL has problems on some platforms.  We're closer to having it work 
# automatically out of the box, but not close enough to enable it by default
UseOpenGL=false

# Color assignment rules
# Keep in order, do not skip numbers. Numbers start
# at 1.
# 
# Pattern:  "Label", "regex", R,G,B, R,G,B
# Label is optional.  If it is omitted, the regex
# will be used.
#
ColorAssign1="Tests",".*test.*", 90,225,90, 110,200,90
ColorAssign2="Build Infrastructure",".*config.*|.*Make.*|.*cfg", 198,229,255, 178,209,235
ColorAssign3="Ports",".*score/cpu.*", 90,225,225, 110,200,200
ColorAssign4="BSPs",".*libbsp.*|.*libcpu.*|.*libchip.*", 30,144,255, 10,124,235
ColorAssign5="Source Code",".*(\.py|\.rb|\.erb|\.hs|\.sql|\.[sSchd]|\.cpp|\.cc|\.m|\.js|\.pl|\.sh|\.java|\.lhs|\.hi|\.ad[abs])", 225,90,90, 200,90,110
ColorAssign6="Documents/Images",".*(\.txt|\.html|\.tex|\.t|\.texinfo|\.texi|\.tmpl|\.css|\.xml|\.yml|\.json|\.png|\.jpg|\.gif|\.jpeg|README|COPYING|LICENSE|AUTHORS)", 90,90,225, 90,110,200

# Save each frame to an image?
TakeSnapshots=true

# Where to save each frame
SnapshotLocation=code_swarm_frames/#####.png

#Is the input xml sorted by date?  It's faster and uses much less memory if it is
IsInputSorted=true
#  - All of the given scripts for producing repository xml files produce sorted data
#  - Please file a bug if one doesn't

# Uncomment to not use avatars
#AvatarFetcher=NoAvatar