Local Automation for repository creation

For the time being, the gecko-strings repository isn’t created in automation, but locally by Axel or flod.

To streamline this, there’s a wrapper script that automates the invocation of initial.cli.create.

The script relies on a couple of mercurial extensions from version-control-tools, and calls create for each push to any of the upstream repositories, in chronological order.

Example:

$> python -m  initial.cli.automate ~/src/ ../output

This expects a clone of mozilla-central under that name in src, along with one of comm-central. Each need to be configured to have at least the following extensions enabled:

[extensions]
firefoxtree = ~/.mozbuild/version-control-tools/hgext/firefoxtree

In addition of the requirements of create (mozxchannel, compare-locales), the automation script depends on mozautomation from version-control-tools/pylib/mozautomation. It’s best practice to create a virtualenv, and to pip install -e . each of those requirements.

The script will pick up the latest version of each upstream repository by inspecting the local meta data in the gecko-strings repository, or by looking at output/latest.json. The json is a local cache of the last revisions handled by the script, independently of whether those actually modified the target repository. It’s only used as an optimization, and is not required.

Review

Once you’ve updated the gecko strings repository, you should review the generated changesets locally. This is best done by keeping a local hg server running by hg serve in output/en-US, and opening the graph view of that in your browser. After the automate script is run, reload the graph to check for new changesets.

It’s a good idea to do this push-by-push, which you can do by passing --interactive to the automate script. That will pause execution whenever the target repository is modified. At that point you can inspect as described below, and even push to quarantine when you’re fine.

When inspecting the graph, the first thing to check is that there’s still a single head of our repository, and that that head is connected to the previous one. Also, that we didn’t create new roots. If either happens, Axel needs to investigate. Please don’t push to the quarantine repository in this state.

After inspecting the graph, inspect each new commit. Things to check for:

• Does the commit meta data look OK? Inspect the X-Channel- data in the description of the commit.
• Does the diff look OK? If there are removed strings, is that a backout? Or a string only in Nightly? Or did we break cross-channel?

To be safe, it’s a good idea to update the local mozilla-central and comm-central clones to the central and beta states, and to run compare-locales like so:

$> compare-locales --full --unified -Dmozilla=$HOME/src/mozilla-central comm-central/mail/locales/l10n.toml output en-US

This should show only obsolete strings, and on central, no changed strings. Changed strings are expected on beta. Run it on all of browser, mobile/android, mail, calendar, and suite. For each of m-c and c-c, it’s OK to do that in one pass.

Step-by-Step

The steps below assume a pristine setup. You should be able to re-use existing clones, though. You will need to make sure to enable the extensions.

cd
mkdir src; cd src
virtualenv venv
source venv/bin/activate
pip install mercurial==5.4.1 requests==2.24.0
cd ~/.mozbuild/version-control-tools
hg pull -u
cd ~/src
hg clone https://hg.mozilla.org/l10n/compare-locales/
for d in ~/.mozbuild/version-control-tools/cross-channel-l10n ~/.mozbuild/version-control-tools/pylib/mozautomation ~/src/compare-locales; do
pip install -e $d
done
hg clone https://hg.mozilla.org/mozilla-central/
hg clone https://hg.mozilla.org/comm-central/
for r in mozilla-central comm-central; do
cat >> $r/.hg/hgrc << EOF

[extensions]
firefoxtree = ~/.mozbuild/version-control-tools/hgext/firefoxtree
EOF
done

cd mozilla-central
for upstream in central beta release; do
hg pull $upstream
done
cd ..

cd comm-central
for upstream in comm comm-beta comm-release; do
hg pull $upstream
done
cd ..

mkdir output
hg clone https://hg.mozilla.org/l10n/gecko-strings/ output/en-US

hg clone https://hg.mozilla.org/users/axel_mozilla.com/cross-channel-experimental/

and now you’re all set up. To run automation, you will do

cd ~/src
source xenv/bin/activate
cd cross-channel-experimental/
python -m  initial.cli.automate ~/src/ ../output