https://github.com/openmicroscopy/snoopycrimecop hosts a series of utility scripts used in the continuous integration.
If you find a bug or if you want an additional feature to be implemented, please open an issue.
The scc tools are a set of Python based utility programs. The tools suite can be installed using pip:
$ pip install -U scc
If your Python is old then this may also install the PyGithub and argparse packages, but since Python 2.7 they are already included with the standard library.
Most of the scc commands instantiate a Github connection using the PyGithub package. Unless the --token option is passed to the scc command, the command first looks for the github.token specified in the git config file and, if found, uses this token to connect to Github:
$ ./scc.py merge master --info -v
2013-01-16 22:03:49,633 [ scc.config] DEBUG Found github.token
...
If no token is found, the command looks for a github.user in the git config file and, if found, uses this username to connect to Github:
$ ./scc.py merge master --info -v
2013-01-16 22:06:00,256 [ scc.config] DEBUG Found github.user
Enter password for http://github.com/sbesson:
Note
The password to be entered here is the Github password.
Finally, if no token or user is found, both the Github username and password are queried at the prompt:
$ ./scc.py merge master --info -v
# github.token and github.user not found.
# See `./scc.py token` for simpifying use.
Username or token: sbesson
Enter password for http://github.com/sbesson:
Merge all the open PRs based on specified branch, including all submodules.
The first argument is the name of the base branch of origin, e.g.:
$ scc merge develop
Add a comment to the PR if there is a conflict while merging the PR
$ scc merge develop --comment
Specify the default set of PRs to be included in the merge
Four options are currently implemented: none, mine, org and all. By default (org), only PRs opened by public members of the repository organization are included so the following are equivalent:
$ scc merge develop
$ scc merge develop --default org
$ scc merge develop -D org
Exclude PR by label, user or number
Label filters can be specified using label:my_label:
$ scc merge develop -E label:exclude_label
User filters can be specified using user:username:
$ scc merge develop -E user:my_user
PR filters can be specified using pr:number or #number:
$ scc merge develop -E pr:45 -E #46
Submodule PR filters can be specified using submodule_name#number:
$ scc merge develop -E bioformats#40
If no key is specified, it is assumed the filter is a label filter.
By default, only PRs labeled as exclude are excluded so the following are equivalent:
$ scc merge develop
$ scc merge develop -E label:exclude
$ scc merge develop -E exclude
Include PR by label, user or number
Label filters can be specified using label:my_label:
$ scc merge develop -I label:exclude_label
User filters can be specified using user:username:
$ scc merge develop -I user:my_user
PR filters can be specified using pr:number or #number:
$ scc merge develop -I pr:45 -I #46
Submodule PR filters can be specified using submodule_name#number:
$ scc merge develop -I bioformats#40
If no key is specified, it is assumed the filter is a label filter.
By default, only PRs labeled as include are included so the following are equivalent:
$ scc merge develop
$ scc merge develop -I label:include
$ scc merge develop -I include
Display the candidate PRs to merge but do not merge them
$ scc merge develop --info
Push the locally merged branch to Github
$ scc merge develop --push my-merged-branch
Recursively reset each repository to the HEAD of the base branch
$ scc merge develop --reset
Merge the PRs for the top-level directory only, excluding submodules:
$ scc merge develop --shallow
Specify the name of the remote to use as the origin. Default: origin:
$ scc merge develop --remote gh
As a concrete example, the first step of the OMERO-merge-develop job is calling the following merge command:
$ scc merge develop --no-ask --reset --comment --push merge/develop/latest
Changed in version 0.3.0: Added default values for --include and --exclude options.
Changed in version 0.3.8: Added --shallow and --remote options.
Merge PRs in a Travis environment, using the PR comments to generate the merge filters.
$ scc travis-merge
This command internally defines all the filter options exposed in scc merge.
The target branch is read from the base of the PR, the --default option is set to none meaning no PR is merged by default and no default --exclude option is defined.
The --include filter is determined by parsing all the PR comments lines starting with --depends-on. To include PR 67 in the merge, add a comment line starting with --depends-on #67 to the PR. To include PR 60 of the bioformats submodules, add a comment line starting with --depends-on bioformats#60 to the PR.
Update the pointer of all submodules based on specified branch.
The first argument is the name of the base branch of origin, e.g.:
$ scc update-submodules develop
Push the locally merged branch to Github and open a PR against the base branch:
$ scc merge develop --push submodules_branch
Combined with --push option, push the locally merged branch to Github but skip PR opening:
$ scc merge develop --push submodules_branch --no-pr
Specify the name of the remote to use as the origin (default: origin):
$ scc update-submodules develop --remote gh
Rebase a PR (open or closed) onto another branch and open a new PR.
The first argument is the number of the PR to rebase and the second argument is the name of the branch onto which the PR should be rebased:
$ scc rebase 142 develop
Assuming the head branch used to open the PR 142 was called branch_142, this command will rebase the tip of branch_142 onto origin/develop, create a new local branch called rebased/develop/branch_142, push this branch to Github and open a new PR. Assuming the command opens PR 150, to facilitate the integration with scc unrebased-prs, a --rebased-to #150 comment is added to PR 142 and a --rebased-from #142 comment is added to the PR 150. Finally, the command will switch back to the original branch prior to rebasing and delete the local rebased/develop/branch_142.
Note
By default, scc rebase uses the branches of the origin remote to rebase the PR. To specify another remote, use the --remote option.
Skip the opening of the PR
$ scc rebase 142 develop --no-pr
Do not delete the local rebased branch
$ scc rebase 142 develop --no-delete
Specify the name of the remote to use for the rebase (default: origin)
$ scc rebase 142 develop --remote snoopycrimecop
Re-run the command after manually fixing conflicts
If scc rebase fails due to conflict during the rebase, you will end up in a detached HEAD state.
If you want to continue the rebase operation, you will need to manually fix the conflicts:
# fix files locally
$ git add conflicting_files # add conflicting files
$ git rebase --continue
This conflict solving operation may need to be repeated multiple times until the branch is fully rebased.
Once all the conflicts are resolved, call scc rebase with the --continue option:
$ scc rebase --continue 142 develop
Depending on the input options, this command will perform all the steps of the rebase command (Github pushing, PR opening) skipping the rebase part.
Alternately, you can abort the rebase and switch to your previous branch:
$ git rebase --abort
$ git checkout old_branch
Changed in version 0.3.10: Automatically add --rebased-to and --rebased-from comments to the source and target PRs.
Compare two development branches and check that PRs merged in one branch have been merged to the other.
The basic workflow of the scc unrebased-prs command is the following:
Additionally, for each line of each PR starting with --rebased-to or --rebased-from, the existence of a matching line is checked in the corresponding source/target PR. For instance, if PR 70 has a --rebased-from #67 line and a --rebased-from #66 line, then both PRs 66 and 67 should have a --rebased-to #70 line.
This command requires two positional arguments corresponding to the name of the branch of origin to compare:
$ scc unrebased-prs dev_4_4 develop
Specify the name of the remote to use as the origin (default: origin):
$ scc unrebased-prs dev_4_4 develop --remote gh
Do not check links between rebased comments:
$ scc unrebased-prs dev_4_4 develop --no-check
New in version 0.3.10: Added support for body/comment parsing and --rebased-to/from linkcheck
Deploy a website update using file symlink replacement:
$ scc deploy folder
The goal of this command is to enable overwriting of deployed doc content and allow for “hot-swapping” content served by Apache without downtime and HTTP 404s.
Prepare folder for symlink replacement. Should only be run once
$ scc deploy folder --init
New in version 0.3.1.
The hudson jobs BIOFORMATS-docs-merge-stable, BIOFORMATS-docs-merge-develop, OMERO-docs-merge-stable, OMERO-docs-merge-develop and OMERO-docs-internal deploy the documentation artifacts to necromancer. The target directory (sphinx-docs) is controlled by the hudson:hudson user, so all file system operations are allowed. Each job has the target directory configured in the SSH publisher target directory property. After deployment has happened to a temporary directory, a series of symlink moves happens making sure that the symlink points to the updated content.