EXPERIMENTS.md

Simulation experiment procedure for ISER 2018

1. Make sure your launcher uses an appropriate exp_prefix parameter (e.g. exp_prefix=trpo_reacher)

2. Always start from master

   cd garage
   git checkout master
   git reset --hard HEAD

3. Modify parameters (only) in launcher file

4. Make a commit to the experiments branch and push to GitHub

   git checkout -b experiments
   git commit -m "Experiment description"
   git push origin experiments  # don't force-push!

5. Run experiment

6. Kill the experiment processes

   garage has an unfortunate bug which leads to run_experiment sometimes leaving zombie processes after termination. To close all the run_experiment processes, you can use this one-liner.

   ps -fu | grep run_experiment | awk '{print $2}' | xargs kill -SIGINT

7. Manually log the git commit hash to the experiment directory

   export EXP_DIR_SUFFIX="my_launcher_prefix/my_experiment"  # copied from run_experiment output
   echo "$(git rev-parse HEAD)" > data/local/$EXP_DIR_SUFFIX/git_hash

8. Copy the log directory (in garage/data/local/<experiment_prefix>/<experiment>) to the GCS bucket:

   gsutil -m rsync -r garage/data/local/$EXP_DIR_SUFFIX gs://resl-iser2018/experiments/$EXP_DIR_SUFFIX

Google Cloud Storage setup

1. Install Google Cloud SDK
2. gcloud init
3. Choose project resl-iser2018
4. Choose zone us-west-1c
5. Test

   gsutil cat gs://resl-iser2018/hello_world.txt

Testing

When testing your scripts or experimenting with gsutil, please use the devel bucket at gs://resl-iser2018-devel to avoid polluting the history of the authoritative bucket.