Upgrading the Tor Browser

The big picture
Upgrade Tor Browser in Tails
1. Communicate
2. Wait
3. Verify
4. Set download url
5. Commit
Sync with the upstream wrapper scripts
Self-hosted Tor Browser tarballs archive

The big picture

The Tails build system downloads a set of Tor Browser tarballs from a location specified in config/chroot local-includes/usr/share/tails/tbb-dist-url.txt, and compares their hash with previously verified ones found in config/chroot local-includes/usr/share/tails/tbb-sha256sums.txt.

Once released officially, Tor Browser tarballs can be found in a permanent (?) location. However, when upgrading Tor Browser for an imminent Tails release, we generally have to use Tor Browser tarballs that are under QA and not officially released yet. So, we have to retrieve them from another, temporary location, such as http://people.torproject.org/~mikeperry/builds/. If we hard-coded this temporary URL in tbb-dist-url.txt, then our release tag would only be buildable for as long the tarballs stay in that place, which at best is a few months.

To solve this, we host ourselves the Tor Browser tarballs we need, and point to this permanent location for anything that we tag.

Still, one can set an arbitrary download location in tbb-dist-url.txt, which should provide all the flexibility needed for development purposes, e.g. when importing an alpha version with no intention to include it in a tagged Tails release, in which case a stable URL is not needed and we can point directly to the upstream URL.

Occasionally, the new Tor Browser requires adjustments to the corresponding AppArmor profile: AppArmor policy.

Upgrade Tor Browser in Tails

Communicate

Early on release-week-Monday, on the Tor Browser dev IRC channel, announce that this week you're the person in charge of monitoring when a new just-trustworthy-enough Tor Browser tarball is ready, of importing it into Tails, and of running it through our QA.

Rationale: establish a rapport so that the Applications team members who are working on the Tor Browser release process have our needs in mind, and match these needs with an individual IRC nick they can keep up-to-date. This increases the chances you'll be kept in the loop at whatever time is relevant to this release process. This also provides context for any question you might need to ask later on, especially in terms of timing & availability of a tarball that's good enough for us to import.

Wait

To check if the desired version is available:

Set TBB_VERSION to the version of Tor Browser you are waiting for, e.g. 42.0.12.

Run this script:

 ```sh
 (
     set -eu
     typeset -a tbbUrls
     tbbUrls=( 'https://archive.torproject.org/tor-package-archive/torbrowser/?C=M;O=D' 'https://www.torproject.org/dist/torbrowser/?C=M;O=D' 'https://people.torproject.org/~'{boklm,gk,ma1,morgan,pierov}'/builds/?C=M;O=D' 'https://tb-build-0'{2,3}'.torproject.org/~'{boklm,ma1,morgan,pierov}'/builds/torbrowser/release/unsigned/'  )
     while true
     do
         if (for u in "${tbbUrls[@]}"; do
                 echo "query ${u}" >&2
             elinks -no-numbering  -dump "$u" | sed -e "s+^+[${u//+/ }]+" ; done
             ) | grep -F -e "${TBB_VERSION?}" -e "$(date +%F)"  -e "$(date +%F -d yesterday)" --color=always
         then
             echo FOUND
             notify-send "Release dir" 'found'
         fi
         sleep 10m
     done
 )
 ```

Verify

Set TBB_DIST_URL to the chosen URL, and set TBB_VERSION to the desired Tor Browser version, for example:

TBB_DIST_URL=https://people.torproject.org/~mikeperry/builds/4.5-build5/
TBB_VERSION="$(basename "${TBB_DIST_URL:?}")"

Ensure you include the "-buildN" part. Add it if it's missing.

Set TBB_TAILS_TICKET to the number of the ticket about updating Tor Browser, and TBB_IMPORT_BRANCH to the branch we'll work on, which must include the -force-all-tests suffix:

TBB_TAILS_TICKET=123456
TBB_IMPORT_BRANCH=${TBB_TAILS_TICKET:?}-tor-browser-${TBB_VERSION%%-*}-force-all-tests

Check out the new branch:

git checkout -b ${TBB_IMPORT_BRANCH:?} origin/stable

Set TBB_SIGNER_SUFFIX to the nickname of the Tor Browser developer who signed the sha256sums-unsigned-build.txt file, preceeded by a dash; the detached signature at $TBB_DIST_URL is named sha256sums-unsigned-build.txt.asc${TBB_SIGNER_SUFFIX}:

TBB_SIGNER_SUFFIX='-boklm'

Note: Nowadays it's quite common that the detached signature is called sha256sums-unsigned-build.txt.asc, with no suffix. If that's the case, just set it to an empty string:

TBB_SIGNER_SUFFIX=''

Fetch the version's hash file and its detached signature, and verify with GnuPG:

(
    set -e
    gpg --refresh-keys EF6E286DDA85EA2A4BA7DE684E2C6E8793298290
    for f in sha256sums-unsigned-build.txt{.asc${TBB_SIGNER_SUFFIX?},}; do
        wget "${TBB_DIST_URL:?}/$f" -O "$f"
    done
    gpg --verify sha256sums-unsigned-build.txt{.asc${TBB_SIGNER_SUFFIX?},}
)

Build the list of tarballs we want and their hashes, so this information is available at build time, when the tarballs are fetched:

(
    grep --color=never -E \
        "\stor-browser-linux-x86_64-${TBB_VERSION%-build*}\.tar\.xz$" \
        sha256sums-unsigned-build.txt
) > config/chroot_local-includes/usr/share/tails/tbb-sha256sums.txt

Set download url

If `TBB_DIST_URL` supports http

Then update the URL to the one chosen above:

echo "${TBB_DIST_URL:?}" | sed "s,^https://,http://," > \
     config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt

We cannot use HTTPS: building Tails uses a caching HTTP proxy by default and there's no way such a proxy caches data without MitM'ing connections. However, it is of no consequence since we verify the checksum file.

Else

Nowadays, tarballs are usually put on a web server, such as https://tb-build-*.torproject.org/, that doesn't support http: it will redirect to https.

If this is the case, you must put those tarballs on a server that allows us to use plain HTTP.

We cannot use such a server directly, so we need to (at least temporarily) self-host this.

If you can access torbrowser-archive.git

echo "${TBB_DIST_URL:?}" > \
     config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt
git commit config/chroot_local-includes/usr/share/tails/tbb-*.txt \
    -m "Upgrade Tor Browser to ${TBB_VERSION:?} (refs: tails#${TBB_TAILS_TICKET})" && \
git show

Please refer to Self-hosted Tor Browser tarballs archive section.

If you can't access it

Download the relevant files:

 (
    grep --color=never -Eo \
        '\stor-browser-linux-x86_64-${TBB_VERSION%-build*}\.tar\.xz$' \
        sha256sums-unsigned-build.txt \
) | sed -re "s@^\s*@${TBB_DIST_URL:?}@" | xargs wget -c

Now, put them on some webserver that you can control. Let's say you uploaded them to http://myserver.net/misc/tbb/. Put that URL in config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt

Commit

Lastly, commit:

git commit config/chroot_local-includes/usr/share/tails/tbb-*.txt \
    -m "Upgrade Tor Browser to ${TBB_VERSION:?} (refs: tails#${TBB_TAILS_TICKET})" && \
git show

If this new Tor Browser is meant to be included in a Tails release, then that's not enough: as explained above, we need to host the corresponding tarballs ourselves, so read on the next section.

Sync with the upstream wrapper scripts

Adapt our config/chroot_local-includes/usr/local/bin/tor-browser and/or config/chroot_local-includes/usr/local/lib/tails-shell-library/tor-browser.sh for recent changes made in the Tor Browser build Git repo (git clone https://gitlab.torproject.org/tpo/applications/tor-browser-build.git), on the maint-X.Y-desktop branch (if it does not exist, fall back to maint-X.Y):

git log -p \
    projects/firefox/abicheck.cc \
    projects/firefox/start-firefox \
    projects/browser/RelativeLink/start-browser

Then apply any relevant change, e.g. to:

environment variables;
commandline options passed to the firefox executable;
required libstdc++6 version bumps; if there's been any change upstream, look for abicheck in config/chroot_local-hooks/10-tbb and adjust that hook as needed.

We also document (as code comments that includes the analysis, in config/chroot_local-includes/usr/local/lib/tails-shell-library/tor-browser.sh) each such environment variable (and similar) that we consider irrelevant and skip, both to avoid others duplicating the analysis and to make it explicit during the review so it can be validated.

Self-hosted Tor Browser tarballs archive

Initial setup

First, install git-annex.

Then, make sure you have an entry for git.tails.net in your ~/.ssh/config. For details, see ISO_history.mdwn in tails/rm.

Then, clone the metadata repository and initialize git-annex:

git clone gitolite@git.tails.net:torbrowser-archive.git && \
cd torbrowser-archive && \
git annex init

You now have a lot of (dangling) symlinks in place of the files that are available in this git-annex repo.

To synchronize your local git-annex metadata with the remote, run:

git annex sync --no-content

Set up environment variables

Make sure you still have the environment variables defined in the previous section set.
Make TAILS_GIT_REPO point to the main Tails Git repository checkout where tbb-dist-url.txt is being worked on, for example:
```
 TAILS_GIT_REPO="$HOME/tails/git"
```
Make TBB_ARCHIVE point to your local git annex working copy of our Tor Browser archive, for example:
```
 TBB_ARCHIVE="$HOME/tails/torbrowser-archive"
```

Import a new set of Tor Browser tarballs

Download and verify all the tarballs we need:

 DL_DIR=$(mktemp --tmpdir -d "tor-browser-${TBB_VERSION}.XXXXXXXXXX")
 CHROOT_INCLUDES="config/chroot_local-includes"
 TBB_SHA256SUMS_FILE="${CHROOT_INCLUDES:?}/usr/share/tails/tbb-sha256sums.txt"
 TBB_DIST_URL_FILE="${CHROOT_INCLUDES:?}/usr/share/tails/tbb-dist-url.txt"
 cd "${TAILS_GIT_REPO:?}" && git checkout "${TBB_IMPORT_BRANCH:?}"
 TBB_TARBALLS_BASE_URL="$(cat "${TBB_DIST_URL_FILE:?}" | sed "s,^http://,https://,")"
 current_branch=$(git -C "${TAILS_GIT_REPO:?}" branch | awk '/^\* / { print $2 }')
 for branch in "$current_branch" ; do
    git -C "${TAILS_GIT_REPO:?}" show "$branch:${TBB_SHA256SUMS_FILE:?}" \
    | while read expected_sha256 tarball; do
       (
          cd "$DL_DIR"
          echo "Retrieving '${TBB_TARBALLS_BASE_URL:?}/${tarball:?}'..."
          curl --remote-name --retry 20 --continue-at - \
             "${TBB_TARBALLS_BASE_URL:?}/${tarball:?}"
       )
    done
    (
       cd "${DL_DIR:?}" && \
       git -C "${TAILS_GIT_REPO:?}" show "$branch:${TBB_SHA256SUMS_FILE:?}" \
          | sha256sum -c -
    )
 done

Import the tarballs into your local Git annex:

 (
 cd "${TBB_ARCHIVE:?}" && \
 mkdir "${TBB_VERSION:?}" && cd "${TBB_VERSION:?}" && \
 git annex import --duplicate "${DL_DIR:?}/"* "${TAILS_GIT_REPO:?}/"sha256sums-*
 )

Commit and push your changes

(
cd "${TBB_ARCHIVE:?}" && \
git commit -m "Add Tor Browser ${TBB_VERSION:?}" && \
git annex sync --no-content && \
git annex copy --to origin -- "${TBB_VERSION:?}" && \
git annex sync --no-content
)

Adjust the URL in the main Git repository

(
cd "${TAILS_GIT_REPO:?}" && \
git checkout "${TBB_IMPORT_BRANCH:?}"
current_branch=$(git branch | awk '/^\* / { print $2 }')
for branch in "${current_branch:?}" ; do
   git checkout "${branch:?}" && \
   echo "http://torbrowser-archive.tails.net/${TBB_VERSION:?}/" > \
        config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt && \
   git commit config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt \
       -m "Fetch Tor Browser from our own archive" && \
   git show
done
)

Wait for the synchronization

Before you can build Tails using Tor Browser from our own archive, you need to wait for the tarballs to appear on https://torbrowser-archive.tails.net/?C=M&O=D. This can take up to 10 minutes:

(
url="$(cat config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt)"
while [ -z "$(elinks -dump -no-references -no-numbering "$url"   | grep 'tor-browser-linux-x86_64.*MiB')" ]
do
    sleep 5
 done
 )

Push the updated branch

git push -o merge_request.create --set-upstream \
   origin "${TBB_IMPORT_BRANCH:?}"

Clean up

(
cd "${TBB_ARCHIVE:?}" && \
git annex drop -- "${TBB_VERSION:?}" && \
git annex sync --no-content && \
rm -rf "${DL_DIR:?}" && \
cd "${TAILS_GIT_REPO:?}" && \
rm -f sha256sums-unsigned-build.txt{,.asc${TBB_SIGNER_SUFFIX}}
)