Contributors guides

Git functionalities

Clone a remote git repository

To clone a remote git repository, simply execute:

$ git clone link/to/repo

This will download the git repository and checkout the main branch.

Set-up username and email

Git won’t let you commit anything while you have not set up the following constants:

  • user.name: the username you will use on the online git repository (also will show up in the commits you make)

  • user.email: your e-mail

Those can be set up globally using:

$ git config --global user.name USERNAME
$ git config --global user.email EMAIL

Or for a specific project by executing the following from the git directory:

$ git config --local user.name USERNAME
$ git config --local user.email EMAIL

Set up mergetool and difftool

git difftool is a utility to show and resolve the differences between two commits. git mergetool is a utility to show and resolve the conflicts between two commits for an ongoing merge.

You can set up once the text editor (such as meld) to use for both utilities using the following commands:

$ git config --global diff.tool EDITOR
$ git config --global merge.tool EDITOR
$ git config --global difftool.prompt false  # disable prompt before executing difftool

Then those commands can be used. If the commands do not work, you may need to customize the configuration of the editor in the .gitconfig file (found in your home directory on unix), depending on the editor you chose.

Example configuration for the editor meld:

# Add the following to your .gitconfig file.
[diff]
    tool = meld
[difftool]
    prompt = false
[difftool "meld"]
    cmd = meld "$LOCAL" "$REMOTE"
[merge]
    tool = meld
[mergetool "meld"]
    # Choose one of these 2 lines (not both!) explained below.
    cmd = meld "$LOCAL" "$MERGED" "$REMOTE" --output "$MERGED"
    cmd = meld "$LOCAL" "$BASE" "$REMOTE" --output "$MERGED"

On the last two lines, the difference is about what version of the merged file you want to display in the centre. $MERGED will contain the partially merged file and the conflict information while $BASE will contain the version before any merging. In both cases, the centre version is the one that will be used upon commiting the merge.

Create feature or hotfix branches

As mentionned previously, you shall work routinely on a temporary feature branch. To create it the first time, checkout first the branch you will use as the base (in our case, dev), then create your branch spanning from it:

$ git checkout dev             # checkout the 'dev' branch
$ git pull                     # update local version of the branch 'dev'
$ git checkout -b n-xxxx       # create and checkout a new branch 'n-xxxx' from current branch ('dev')

Same case when working on a hotfix (see issue tracker).

Keep local branches up-to-date

There are multiple ways to keep your git local repository up-to-date with the online repository origin. The simplest way if you are well aware of the git structure or are retrieving changes on a branch you have not locally modified is to execute a pull:

$ git checkout BRANCH_NAME  # checkout `BRANCH_NAME` you want to update
$ git pull                  # pull the changes from the online repository `origin`

If you are unsure, you can first fetch the latest version of every branch: git fetch This command will fetch the branches on all online repositories configured (in our case only origin). You can then check the state of the branches using a utility like gitg. To apply the changes of the remote version of a branch to your local version, you can merge them:

$ git fetch                         # fetch the list of changes from the repositories (without applying them)
$ git checkout BRANCH_NAME          # checkout `BRANCH_NAME` you want to update
$ git merge REMOTE_NAME/BRANCH_NAME # merge the changes from the online repository REMOTE_NAME (e.g 'origin') onto the branch BRANCH_NAME

Work on feature or hotfix branch

When working, you should commit and push your changes as often as possible. To do that, you should select the changes you want to commit using any of the following:

  • git add FILENAME: add all changes made to FILENAME since last commit (or add FILENAME to the tracked files if it wasn’t before)

  • git add FOLDER_NAME: add all changes and all filenames contained in FOLDER_NAME

  • git add -u: add all changes made to tracked files

  • git add -p [OTHER_OPTIONS]: add all changes interactively, you can select precisely which chunks of code are being commited

  • git add --all: add all changes and files to match the current working tree (Not advised except on the initial commit of a new repository)

  • git mv FILENAME NEW_FILENAME: move/rename the filename FILENAME to NEW_FILENAME

  • git rm [--cached] FILENAME: remove filename FILENAME from the tracked files (as well as from the current directory if the option --cached is not used)

You can check the status of your git branch at any time: git status

If you want to abort your commit operation, you can execute: git reset HEAD (Note: it will mess up the git mv operations as the files will keep their new name)

When you are satisifed with the changes you have selected, you can perform the commit:

git commit -m COMMIT_MESSAGE [-m COMMIT_DETAILS] [-m ISSUE_REFS] (see this for help in formatting your commits)

If you have any second thought about your performed commits, you can easily undo them while they have not been commited with: git reset HEAD~ (will cancel the last unpushed commit)

Once your commit performed, if you are happy with the changes, push them:

$ git push origin LOCAL_BRANCH_NAME``  # (for example: `git push origin 1-add-electre-4`)

Rebase of your branch

Warning

Do not do the following if you don’t have a solid understanding of git in general and the branches structure of the repository you work on!

When you are working on your branch, and changes are being made on the common branch you are based on (e.g dev) but you are not ready to merge your changes, or you want your changes to take into account the changes of the common branch retrospectively, you could perform a rebase of your branch on the common branch:

$ git checkout dev           # checkout 'dev' branch
$ git pull                   # make sure the local version of 'dev' is up-to-date with 'origin' repository
$ git checkout n-xxxx        # feature branch we want to rebase
$ git rebase dev             # modify branch 'n-xxxx' history so its unmerged changes appear after the last `dev` commits

After performing a rebase, you may have to force the next push you will make of this branch as the online repository will have to also modify previous history to cope with the rebase:

$ git push --force origin n-xxxx

Warning

Never force push on a common branch as it will override the remote history of the branch

However this rebase operation modifies the history of the branch on which we are rebasing, so it must never be performed on any common branch. This would otherwise force all other users with changes spanning from this common branch to update their whole git history, potentially creating conflicts in their previous commits.

Merge changes on common branch

When you have completed the implementation of a new feature, or made any significant change to the code and you feel confident enough to add them on the common dev branch, you should consider merging the branches. Remember to check you have updated the changelog file, adding the new feature(s) you developed to it.

First, make sure you have commited and pushed your changes on your working branch. Then, it is important to make sure you get the up-to-date version of the branch on which you want to merge (e.g dev).

$ git checkout dev # checkout the 'dev' branch
$ git pull         # make sure the local version of 'dev' is up-to-date with 'origin' repository

You are ready to merge.

You may encounter some conflicts as the dev branch may have had some changes since you last split from it or merged to it. In order to limit those conflicts and keep the git history as clean as possible, you can rebase your branch on the common branch, to take into account any changes into any common branch that has changed since your last merge/split, before merging. In this case, follow the section on rebase.

To perform the actual merge:

$ git checkout dev                   # checkout branch 'dev' where we want to merge 'n-xxxx'
$ git merge [MODE] n-xxxx            # merge branch 'n-xxxx' into 'dev'

There are different modes to perform a merge:

  • --ff (default mode): will attempt fast-forward merge if possible, fail back on a 3-way merge if conflicts are detected

  • --ff-only: will only perform merge if no conflicts are detected

  • --no-ff: will always perform a 3-way merge

If any conflicts appear, the merge will be suspended. You can then resolve the conflicts either by hand by modifying directly the files reported to have conflicts, or by using the command:

$ git mergetool

The latter will open the configured editor to ease the resolution of the conflicts, showing one file at a time, both conflicting versions on the sides, and the merged version in the centre. Once you have resolved all conflicts, you can create a commit for the merge:

$ git commit -m MESSAGE

If you want to abort the merge, you can execute:

$ git merge --abort

When you have finished merging your changes on any common branch, you should push them as soon as possible. Any delay before pushing changes increases the chances of conflicts for you and the other persons working on this branch. If you are a developer and not a maintainer, pushing a merge on a common branch will prompt you to issue a merge request. This merge request will freeze the push until a maintainer manually validate the changes brought about by this push. To access the merge request associated with your merge operation, you can open the link returned to you by the push in the terminal. Alternatively, you can merge your branch and issue a merge request at the same time on gitlab website by creating a new merge request, specifying the branches to merge.

Note

  • you can ask the merge request to delete the branch being merged on the common branch (default behaviour)

  • we do not recommend stashing all changes in one commit when merging

When you merge a feature branch, it should mean the feature implementation is over. You may then want to delete the local version of this branch in order to keep your number of local branches as small as possible:

$ git branch -d BRANCH_NAME                # delete local branch 'BRANCH_NAME'
$ git branch -d -r REPOSITORY/BRANCH_NAME  # delete remote tracking of 'BRANCH_NAME' from repository 'REPOSITORY' (e.g 'origin')

Repurpose an outdated branch

If for some reasons, you want to work on a branch which is completely outdated but the branch was not attached to an up-to-date branch, and you don’t care about the last unmerged commits. You basically want to create a new branch with the same name, then you should delete the existing branch and create a new one from any commit/branch you want:

$ git branch -d BRANCH_NAME  # delete local version of the branch
$ git checkout XXXX          # checkout a branch or a specific commit as the new starting point
$ git switch -c BRANCH_NAME  # create and switch to a new branch named `BRANCH_NAME` starting from `XXXX`

If you need to push changes to this branch, you should first delete the remote version of the branch, then push your local branch on the remote:

$ git push origin --delete BRANCH_NAME  # delete 'origin' remote version of the branch
$ git push origin BRANCH_NAME           # push local version of the branch on the remote 'origin'

Note

When deleting a remote version of a branch, the previous commits done on this branch will remain on the remote, they will simply not be associated to the branch name anymore.

Release a new version

When the development done on dev branch is stable enough, and significant changes have been made, we can release an new version of the code on the git master branch.

It simply requires to merge the dev branch on the master branch, and then tag this new release with a version number (do not forget to update the changelog and requirements beforehand):

$ git checkout master    # checkout the 'master' branch
$ git merge --no-ff dev  # merge the 'dev' branch inside in 3-way mode
$ git tag VERSION        # tag the new release with VERSION

If the release process necessitates other steps, you should create a temporary release branch release/VERSION (VERSION indicating the version being released). You will them perform the steps necessary for releasing the new version on this branch. After the release is complete, you will merge it on the master branch, tag it, then merge it on dev, then delete the release branch.

$ git checkout dev                         # checkout the 'dev' branch which you want to release
$ git checkout -b release/VERSION          # create a release branch and checkout it
...                                      # work on the release process
$ git checkout master                      # checkout 'master' branch
$ git merge --no-ff release/VERSION        # merge the release branch inside in 3-way mode
$ git tag VERSION                          # tag the new release with VERSION
$ git checkout dev                         # checkout 'master' branch
$ git merge --no-ff release/VERSION        # merge the release branch inside in 3-way mode
$ git branch -d release/VERSION            # delete the local release branch
$ git push origin --delete release/VERSION # delete the remote release branch (only if it was pushed)

As you can see, the merge for a release is made in 3-way mode. As a matter of fact, never merge on the master branch in fast-forward as it must only contain the commits of each release. Also any changes on the master branch must come from either the dev branch or an hotfix branch (see issue tracker).

Python environments

While you can use the default system-wide python environment when working on a single project, although you can have permission issues due to you system install, when working on multiple projects with different requirements (packages, package versions, python version) you can bump into a bunch of compatibility issues.

In order to limit the possible conflicts with any other projects, it is recommended you set up pyenv and a pyenv-virtualenv or any other mean of containing this project and keep it separate. You can follow the installation instructions of pyenv, then pyenv-virtualenv.

This will enable you to install multiple versions of python and set up virtual environments which contain their own python interpreter and their own python packages. This way, projects may coexist side-by-side, each with their own set of installed packages and their python version.

Our linters

If you want to know how each linter work, read the following.

flake8

Usage: flake8 [OPTIONS] root_src_folder

Recommended options:

  • --max-line-length n: set maximum line length to n (79 in our case)

  • --ignore=rule_1,rule_2,...: ignore the given set of rules (recommend E203,W503 for black compatibility)

isort

Usage: isort [OPTIONS] root_source_folder

Options when applying changes:

  • --atomic: to avoid syntax errors when applying changes

Options for checking for changes to be made:

  • --diff: print the changes to be made instead of applying them directly

  • --check: check the files for unsorted/unformatted imports and returns boolean result (can be used atop --diff)

Options to use in any situation:

  • --profile black: to make isort operations compatible with black

  • --combine-star: ensure that if a star import is present, nothing else is imported from that namespace

  • --use-parentheses: use parenthesis for line continuation instead of slashes

  • -m VERTICAL_HANGING_INDENT: organize multi-line imports so only one import per line is used

  • -w n: set maximum line length to n (as already said, we recommend 79)

black

Usage: black [OPTIONS] root_src_folder

Option when checking for changes to be made:

  • --check

Option to use in any situation:

  • --line-length n: set maximum line length to n (as already said, we recommend 79)

Note

For many IDE, black can be directly integrated to auto-format the code.

mypy

Usage: mypy [OPTIONS] root_src_folder

Options for using external packages without type checking:

  • --ignore-missing-imports: ignore imported modules which have no type hinting stubs

Implementation recipes

Split implementation/user modules

When adding a new feature, it must be implemented as a part of the mcda.internal subpackage. Let’s say, we implement the following module:

# src/mcda/internal/core/my_super_module.py

from .utils import set_module

@set_module()
class MySuperClass:
    pass

def my_super_function():
    pass

def my_internal_function():
    pass

Then this feature must be placed in the most relevant user module. This is done by using the mcda.internal.core.utils.set_module() decorator on classes implementation, as well as import nominally the functionality in the user module. Indeed this will tell sphinx and python the class location in the user API (so links in the doc work on the class when used as type hint). Also, the functionality must be appended to the user module __all__ variable so that sphinx autodoc shows it (unnecessary for usage though). In our example, let’s create the user module:

# src/mcda/my_super_module.py

from ..internal.core.my_super_module import MySuperClass, my_super_function

__all__ = [
   "MySuperClass",
   "my_super_function"
]

Note

The addition of the features in __all__ is redundant and will be removed once a fix is found for showing imported structures in the doc without it.

Breaking changes in dependency

Some breaking changes may occur in some of the dependencies which affect the package. In this case, we must choose between 3 possible solutions:

  • reject the changes by putting a maximum version number on the dependency in the package pyproject.toml file: this would prevent users/contributors from using a newer version of this dependency

  • take the changes and drop support for previous dependency versions by putting a minimum version number on the dependency in the package pyproject.toml file: this would prevent users/contributors from using a newer version of this dependency

  • enable usage of pre- and post-breaking change dependency versions by creating a wrapper on the changed feature which adapts to both cases based on the dependency version and use it across the package implementation

All compatibility wrappers and tweaks are gathered in the mcda.internal.core.compatibility module and make use of the mcda.internal.core.utils.package_version() function to check the dependency version in question.

Let’s see an example of compatibility wrappers: imagine the dependency dependency package function dependency.function changes its name to f between the versions 1.* and 2.*:

from .utils import package_version

if package_version("dependency") >= (2, 0):
    from dependency import f
    def dependency_function(*args, **kwargs):
        return f(*args, **kwargs)
else:
    from dependency import function
    def dependency_function(*args, **kwargs):
        return function(*args, **kwargs)

We may mark the whole code as uncovered for now. Until we find a more elegnat way to deal with those regarding code coverage.

To see an actual example, look at mcda.internal.core.compatibility.dataframe_map() which addresses the breaking change coming to pandas 3.0 with pandas.DataFrame.applymap() method (currently deprecated) being renamed to pandas.DataFrame.map().

Changes brought by newer python versions

Newer versions of python bring some changes that may simplify the implementation of this package if applied. You can see a list of such changes in Python version subsequent changes. As those changes are incompatible with earlier python versions, we can delay them until we drop support for the incompatible python version in question, or we can differentiate our implementation based on the user installed python version (not always possible).

In the former case, we only need to identify the changes, and list them in the Python version subsequent changes section. In the latter case, we can use this structure:

import sys

if sys.version_info >= (3, 11):  # pragma: nocover
    # code applying to 3.11+ python versions
else:
    # code applying to older python versions

In which case, we mark as uncovered the section of the code which is not run by the development python version.

You can see an example of that with the import of Self from typing_extensions external package for python<3.11 and from typing in python>=3.11.

Deprecation recipes

Replace function/class by a new one

from deprecated.sphinx import deprecated, versionadded

@deprecated(
    reason=
      "It is too inefficient, use new function"
      ":func:`utils.super_module.new_foo`",
    version="0.1.0",
)
def foo():
    """Foo"""
    print("Yolooooooooo!")

@versionadded(
    reason="Old function was inefficient",
    version="0.1.0",
)
def new_foo():
    print("Let's get serious!")

Rename function

from deprecated.sphinx import deprecated, versionchanged

@deprecated(
    reason="Function changed name to :func:`utils.super_module.new_bar`",
    version="0.1.0",
)
def foo():
    return bar()

@versionchanged(
    reason="Function changed with a better name",
    version="0.1.0",
)
#### PUT HERE ANY PREVIOUS DEPRECATION DECORATORS OF OLD FUNCTION
def bar():
    print("Hello I have a fancy name.")

Rename class

from deprecated.sphinx import deprecated, versionchanged

@versionchanged(
    reason="Class changed with a better name",
    version="0.2.0",
)
class B:
    def __init__(self, a):
        self.a = a


@deprecated(
    reason="Class changed name to :func:`utils.super_module.B`",
    version="0.2.0",
)
class A(B):
    pass

Change function/class API

import warnings

from deprecated.sphinx import versionchanged

@versionchanged(reason="API change", version="0.2.0")
def super_print(*args):
    match args:
        case prefix, text:
            _new_super_print(prefix, text)
        case text:
            warnings.warn("API changed!", FutureWarning)
            _old_super_print(text)


def _old_super_print(text):
    _new_super_print("Super", text)


def _new_super_print(prefix, text):
    print(f"{prefix} {text}")

Change function/class behaviour

from deprecated.sphinx import deprecated, versionchanged

@versionchanged(reason="Behaviour has changed!", version="1.0")
def operation(a, b):
    """Future behaviour"""
    return a + b


@deprecated(reason="Behaviour will change", version="0.2.3")
def operation(a, b):
    """Legacy behaviour"""
    return a * b