The Virtual Commons

building extensible software for collective action research

Development News

running vcweb experiments

Experimenter workflow

  1. Create an experiment from the available experiment types (or clone an existing experiment).
  2. Configure the experiment as needed, modifying round durations and other experiment-specific configuration parameters (this won't be implemented until later, but for the time being a select few experimenters can be given access to Django's admin interface)
  3. Add / invite participants to the experiment via mass emails. For controlled experiments in the lab, this can be done transparently via the participant tracker (to be implemented later as well)
  4. Start the experiment when ready, sending all connected participants to an experiment-specific instructions page.
  5. Monitor the experiment and provide explicit input when needed. This will be described in more detail below.

Experimenter driven with timed rounds

We are first implementing support for controlled experiments. These can be experimenter driven at times, with active rounds that have a specific timed duration. Each round has an associated RoundConfiguration that specifies:

  1. whether or not the round should go indefinitely until explicitly signalled by the experimenter (e.g., a checkpoint round).
  2. what type of round it is (current types include quiz rounds, chat rounds, instruction rounds, and normal play rounds.
  3. a integer duration, specified in seconds. If a round has a duration of 0, the round will continue indefinitely until signaled by the experimenter (even if the checkpoint flag is set to false).
  4. an arbitrary number of experiment-specific RoundParameter key-value pairs.
  5. various other book-keeping metadata (creation date, created by, sequence number

Experimenter monitored with long-lived, spaced out rounds

This is to support long-lived experiments that go on over weeks. Each round might be one day or one hour, where participants must contribute to some collective action issue. Will fill in implementation details here later.

Participant workflow

  1. When participants first login to an experiment that hasn't started, they should be ferried to a general instructions welcome screen that acts as a waiting room of sorts.
  2. After the experiment is started by the experimenter, they can progress to the next experiment-specific instructions screen (NOTE: needs server push or constant client polling)
  3. After this, the participant will engage in whatever sequence of round configurations have been specified for the given experiment.

More to come later.

experiment scheduling


The vcweb framework needs to be able to support flexible scheduling of experiments which can run over the course of a month, or in the course of an hour within a controlled computer lab environment. In the latter case a typical experiment run involves a combination of timed rounds where participants make decisions via the web interface, and untimed rounds where participants read instructions, debriefings, or answer survey / quiz questions and only move on when the experimenter has made sure that everyone is on the same page. In order to support timed rounds for controlled settings AND long-running experiments, we need some way to signal our web application that X amount of time has elapsed or that the given long-running experiment round (say a 24-hour round) has completed and that we should now execute our custom experiment-specific logic to calculate results and prep the data needed as input for the next round (or that the experiment is now over, for instance).