Contributors guides
Git functionalities
Clone a remote git repository
To clone a remote git repository, simply execute:
$ git clone link/to/repo
This will download the git repository and checkout the main branch.
Set-up username and email
Git won’t let you commit anything while you have not set up the following constants:
user.name
: the username you will use on the online git repository (also will show up in the commits you make)user.email
: your e-mail
Those can be set up globally using:
$ git config --global user.name USERNAME
$ git config --global user.email EMAIL
Or for a specific project by executing the following from the git directory:
$ git config --local user.name USERNAME
$ git config --local user.email EMAIL
Set up mergetool and difftool
git difftool
is a utility to show and resolve the differences
between two commits. git mergetool
is a utility to show and resolve
the conflicts between two commits for an ongoing merge.
You can set up once the text editor (such as meld
) to use for both
utilities using the following commands:
$ git config --global diff.tool EDITOR
$ git config --global merge.tool EDITOR
$ git config --global difftool.prompt false # disable prompt before executing difftool
Then those commands can be used. If the commands do not work, you may
need to customize the configuration of the editor in the .gitconfig
file (found in your home directory on unix), depending on the editor you
chose.
Example configuration for the editor meld
:
# Add the following to your .gitconfig file.
[diff]
tool = meld
[difftool]
prompt = false
[difftool "meld"]
cmd = meld "$LOCAL" "$REMOTE"
[merge]
tool = meld
[mergetool "meld"]
# Choose one of these 2 lines (not both!) explained below.
cmd = meld "$LOCAL" "$MERGED" "$REMOTE" --output "$MERGED"
cmd = meld "$LOCAL" "$BASE" "$REMOTE" --output "$MERGED"
On the last two lines, the difference is about what version of the
merged file you want to display in the centre. $MERGED
will contain
the partially merged file and the conflict information while $BASE
will contain the version before any merging. In both cases, the centre
version is the one that will be used upon commiting the merge.
Create feature or hotfix branches
As mentionned previously, you shall work routinely on a temporary
feature branch. To create it the first time, checkout first the branch
you will use as the base (in our case, dev
), then create your branch
spanning from it:
$ git checkout dev # checkout the 'dev' branch
$ git pull # update local version of the branch 'dev'
$ git checkout -b n-xxxx # create and checkout a new branch 'n-xxxx' from current branch ('dev')
Same case when working on a hotfix (see issue tracker).
Keep local branches up-to-date
There are multiple ways to keep your git local repository up-to-date
with the online repository origin
. The simplest way if you are
well aware of the git structure or are retrieving changes on a branch
you have not locally modified is to execute a pull:
$ git checkout BRANCH_NAME # checkout `BRANCH_NAME` you want to update
$ git pull # pull the changes from the online repository `origin`
If you are unsure, you can first fetch the latest version of every
branch: git fetch
This command will fetch the branches on all online
repositories configured (in our case only origin
). You can then
check the state of the branches using a utility like gitg
. To apply
the changes of the remote version of a branch to your local version, you
can merge them:
$ git fetch # fetch the list of changes from the repositories (without applying them)
$ git checkout BRANCH_NAME # checkout `BRANCH_NAME` you want to update
$ git merge REMOTE_NAME/BRANCH_NAME # merge the changes from the online repository REMOTE_NAME (e.g 'origin') onto the branch BRANCH_NAME
Work on feature or hotfix branch
When working, you should commit and push your changes as often as possible. To do that, you should select the changes you want to commit using any of the following:
git add FILENAME
: add all changes made toFILENAME
since last commit (or addFILENAME
to the tracked files if it wasn’t before)git add FOLDER_NAME
: add all changes and all filenames contained inFOLDER_NAME
git add -u
: add all changes made to tracked filesgit add -p [OTHER_OPTIONS]
: add all changes interactively, you can select precisely which chunks of code are being commitedgit add --all
: add all changes and files to match the current working tree (Not advised except on the initial commit of a new repository)git mv FILENAME NEW_FILENAME
: move/rename the filenameFILENAME
toNEW_FILENAME
git rm [--cached] FILENAME
: remove filenameFILENAME
from the tracked files (as well as from the current directory if the option--cached
is not used)
You can check the status of your git branch at any time: git status
If you want to abort your commit operation, you can execute:
git reset HEAD
(Note: it will mess up the git mv
operations as
the files will keep their new name)
When you are satisifed with the changes you have selected, you can perform the commit:
git commit -m COMMIT_MESSAGE [-m COMMIT_DETAILS] [-m ISSUE_REFS]
(see this for help in
formatting your commits)
If you have any second thought about your performed commits, you can
easily undo them while they have not been commited with:
git reset HEAD~
(will cancel the last unpushed commit)
Once your commit performed, if you are happy with the changes, push them:
$ git push origin LOCAL_BRANCH_NAME`` # (for example: `git push origin 1-add-electre-4`)
Rebase of your branch
Warning
Do not do the following if you don’t have a solid understanding of git in general and the branches structure of the repository you work on!
When you are working on your branch, and changes are being made on the
common branch you are based on (e.g dev
) but you are not ready to
merge your changes, or you want your changes to take into account the
changes of the common branch retrospectively, you could perform a rebase
of your branch on the common branch:
$ git checkout dev # checkout 'dev' branch
$ git pull # make sure the local version of 'dev' is up-to-date with 'origin' repository
$ git checkout n-xxxx # feature branch we want to rebase
$ git rebase dev # modify branch 'n-xxxx' history so its unmerged changes appear after the last `dev` commits
After performing a rebase, you may have to force the next push you will make of this branch as the online repository will have to also modify previous history to cope with the rebase:
$ git push --force origin n-xxxx
Warning
Never force push on a common branch as it will override the remote history of the branch
However this rebase operation modifies the history of the branch on which we are rebasing, so it must never be performed on any common branch. This would otherwise force all other users with changes spanning from this common branch to update their whole git history, potentially creating conflicts in their previous commits.
Merge changes on common branch
When you have completed the implementation of a new feature, or made any
significant change to the code and you feel confident enough to add them
on the common dev
branch, you should consider merging the branches.
Remember to check you have updated the changelog file,
adding the new feature(s) you developed to it.
First, make sure you have commited and pushed your changes on your
working branch. Then, it is important to make sure you get the
up-to-date version of the branch on which you want to merge (e.g
dev
).
$ git checkout dev # checkout the 'dev' branch
$ git pull # make sure the local version of 'dev' is up-to-date with 'origin' repository
You are ready to merge.
You may encounter some conflicts as the dev
branch may have had some
changes since you last split from it or merged to it. In order to limit
those conflicts and keep the git history as clean as possible, you can
rebase your branch on the common branch, to take into account any
changes into any common branch that has changed since your last
merge/split, before merging. In this case, follow the section on
rebase.
To perform the actual merge:
$ git checkout dev # checkout branch 'dev' where we want to merge 'n-xxxx'
$ git merge [MODE] n-xxxx # merge branch 'n-xxxx' into 'dev'
There are different modes to perform a merge:
--ff
(default mode): will attempt fast-forward merge if possible, fail back on a 3-way merge if conflicts are detected--ff-only
: will only perform merge if no conflicts are detected--no-ff
: will always perform a 3-way merge
If any conflicts appear, the merge will be suspended. You can then resolve the conflicts either by hand by modifying directly the files reported to have conflicts, or by using the command:
$ git mergetool
The latter will open the configured editor to ease the resolution of the conflicts, showing one file at a time, both conflicting versions on the sides, and the merged version in the centre. Once you have resolved all conflicts, you can create a commit for the merge:
$ git commit -m MESSAGE
If you want to abort the merge, you can execute:
$ git merge --abort
When you have finished merging your changes on any common branch, you
should push them as soon as possible. Any delay before pushing
changes increases the chances of conflicts for you and the other persons
working on this branch. If you are a developer and not a maintainer,
pushing a merge on a common branch will prompt you to issue a merge
request. This merge request will freeze the push until a maintainer
manually validate the changes brought about by this push. To access the
merge request associated with your merge operation, you can open the
link returned to you by the push in the terminal. Alternatively, you can
merge your branch and issue a merge request at the same time on
gitlab
website by creating a new merge request, specifying the
branches to merge.
Note
you can ask the merge request to delete the branch being merged on the common branch (default behaviour)
we do not recommend stashing all changes in one commit when merging
When you merge a feature branch, it should mean the feature implementation is over. You may then want to delete the local version of this branch in order to keep your number of local branches as small as possible:
$ git branch -d BRANCH_NAME # delete local branch 'BRANCH_NAME'
$ git branch -d -r REPOSITORY/BRANCH_NAME # delete remote tracking of 'BRANCH_NAME' from repository 'REPOSITORY' (e.g 'origin')
Repurpose an outdated branch
If for some reasons, you want to work on a branch which is completely outdated but the branch was not attached to an up-to-date branch, and you don’t care about the last unmerged commits. You basically want to create a new branch with the same name, then you should delete the existing branch and create a new one from any commit/branch you want:
$ git branch -d BRANCH_NAME # delete local version of the branch
$ git checkout XXXX # checkout a branch or a specific commit as the new starting point
$ git switch -c BRANCH_NAME # create and switch to a new branch named `BRANCH_NAME` starting from `XXXX`
If you need to push changes to this branch, you should first delete the remote version of the branch, then push your local branch on the remote:
$ git push origin --delete BRANCH_NAME # delete 'origin' remote version of the branch
$ git push origin BRANCH_NAME # push local version of the branch on the remote 'origin'
Note
When deleting a remote version of a branch, the previous commits done on this branch will remain on the remote, they will simply not be associated to the branch name anymore.
Release a new version
When the development done on dev
branch is stable enough, and
significant changes have been made, we can release an new version of the
code on the git master
branch.
It simply requires to merge the dev
branch on the master
branch,
and then tag this new release with a
version number (do not
forget to update the changelog and
requirements beforehand):
$ git checkout master # checkout the 'master' branch
$ git merge --no-ff dev # merge the 'dev' branch inside in 3-way mode
$ git tag VERSION # tag the new release with VERSION
If the release process necessitates other steps, you should create a
temporary release branch release/VERSION
(VERSION
indicating the
version being released). You will them perform the steps
necessary for releasing the new version on this branch. After the
release is complete, you will merge it on the master
branch, tag it,
then merge it on dev
, then delete the release branch.
$ git checkout dev # checkout the 'dev' branch which you want to release
$ git checkout -b release/VERSION # create a release branch and checkout it
... # work on the release process
$ git checkout master # checkout 'master' branch
$ git merge --no-ff release/VERSION # merge the release branch inside in 3-way mode
$ git tag VERSION # tag the new release with VERSION
$ git checkout dev # checkout 'master' branch
$ git merge --no-ff release/VERSION # merge the release branch inside in 3-way mode
$ git branch -d release/VERSION # delete the local release branch
$ git push origin --delete release/VERSION # delete the remote release branch (only if it was pushed)
As you can see, the merge for a release is made in 3-way mode. As a
matter of fact, never merge on the master
branch in fast-forward
as it must only contain the commits of each release. Also any
changes on the master
branch must come from either the dev
branch or an hotfix branch (see issue tracker).
Python environments
While you can use the default system-wide python environment when working on a single project, although you can have permission issues due to you system install, when working on multiple projects with different requirements (packages, package versions, python version) you can bump into a bunch of compatibility issues.
In order to limit the possible conflicts with any other projects, it is recommended you set up pyenv and a pyenv-virtualenv or any other mean of containing this project and keep it separate. You can follow the installation instructions of pyenv, then pyenv-virtualenv.
This will enable you to install multiple versions of python and set up virtual environments which contain their own python interpreter and their own python packages. This way, projects may coexist side-by-side, each with their own set of installed packages and their python version.
Our linters
If you want to know how each linter work, read the following.
flake8
Usage: flake8 [OPTIONS] root_src_folder
Recommended options:
--max-line-length n
: set maximum line length ton
(79 in our case)--ignore=rule_1,rule_2,...
: ignore the given set of rules (recommend E203,W503 for black compatibility)
isort
Usage: isort [OPTIONS] root_source_folder
Options when applying changes:
--atomic
: to avoid syntax errors when applying changes
Options for checking for changes to be made:
--diff
: print the changes to be made instead of applying them directly--check
: check the files for unsorted/unformatted imports and returns boolean result (can be used atop--diff
)
Options to use in any situation:
--profile black
: to make isort operations compatible with black--combine-star
: ensure that if a star import is present, nothing else is imported from that namespace--use-parentheses
: use parenthesis for line continuation instead of slashes-m VERTICAL_HANGING_INDENT
: organize multi-line imports so only one import per line is used-w n
: set maximum line length ton
(as already said, we recommend 79)
black
Usage: black [OPTIONS] root_src_folder
Option when checking for changes to be made:
--check
Option to use in any situation:
--line-length n
: set maximum line length ton
(as already said, we recommend 79)
Note
For many IDE, black can be directly integrated to auto-format the code.
mypy
Usage: mypy [OPTIONS] root_src_folder
Options for using external packages without type checking:
--ignore-missing-imports
: ignore imported modules which have no type hinting stubs
Implementation recipes
Split implementation/user modules
When adding a new feature, it must be implemented as a part of the mcda.internal
subpackage.
Let’s say, we implement the following module:
# src/mcda/internal/core/my_super_module.py
from .utils import set_module
@set_module()
class MySuperClass:
pass
def my_super_function():
pass
def my_internal_function():
pass
Then this feature must be placed in the most relevant user module.
This is done by using the mcda.internal.core.utils.set_module()
decorator
on classes implementation, as well as import nominally the functionality in the user module.
Indeed this will tell sphinx and python the class
location in the user API (so links in the doc work on the class when used as type hint).
Also, the functionality must be appended to the user module __all__ variable
so that sphinx autodoc shows it (unnecessary for usage though).
In our example, let’s create the user module:
# src/mcda/my_super_module.py
from ..internal.core.my_super_module import MySuperClass, my_super_function
__all__ = [
"MySuperClass",
"my_super_function"
]
Note
The addition of the features in __all__ is redundant and will be removed once a fix is found for showing imported structures in the doc without it.
Breaking changes in dependency
Some breaking changes may occur in some of the dependencies which affect the package. In this case, we must choose between 3 possible solutions:
reject the changes by putting a maximum version number on the dependency in the package pyproject.toml file: this would prevent users/contributors from using a newer version of this dependency
take the changes and drop support for previous dependency versions by putting a minimum version number on the dependency in the package pyproject.toml file: this would prevent users/contributors from using a newer version of this dependency
enable usage of pre- and post-breaking change dependency versions by creating a wrapper on the changed feature which adapts to both cases based on the dependency version and use it across the package implementation
All compatibility wrappers and tweaks are gathered in the
mcda.internal.core.compatibility
module and make use of the
mcda.internal.core.utils.package_version()
function to check the
dependency version in question.
Let’s see an example of compatibility wrappers: imagine the dependency dependency package function dependency.function changes its name to f between the versions 1.* and 2.*:
from .utils import package_version
if package_version("dependency") >= (2, 0):
from dependency import f
def dependency_function(*args, **kwargs):
return f(*args, **kwargs)
else:
from dependency import function
def dependency_function(*args, **kwargs):
return function(*args, **kwargs)
We may mark the whole code as uncovered for now. Until we find a more elegnat way to deal with those regarding code coverage.
To see an actual example, look at
mcda.internal.core.compatibility.dataframe_map()
which addresses the
breaking change coming to pandas 3.0 with pandas.DataFrame.applymap()
method (currently deprecated) being renamed to pandas.DataFrame.map()
.
Changes brought by newer python versions
Newer versions of python bring some changes that may simplify the implementation of this package if applied. You can see a list of such changes in Python version subsequent changes. As those changes are incompatible with earlier python versions, we can delay them until we drop support for the incompatible python version in question, or we can differentiate our implementation based on the user installed python version (not always possible).
In the former case, we only need to identify the changes, and list them in the Python version subsequent changes section. In the latter case, we can use this structure:
import sys
if sys.version_info >= (3, 11): # pragma: nocover
# code applying to 3.11+ python versions
else:
# code applying to older python versions
In which case, we mark as uncovered the section of the code which is not run by the development python version.
You can see an example of that with the import of Self
from
typing_extensions
external package for python<3.11 and from typing
in python>=3.11.
Deprecation recipes
Replace function/class by a new one
from deprecated.sphinx import deprecated, versionadded
@deprecated(
reason=
"It is too inefficient, use new function"
":func:`utils.super_module.new_foo`",
version="0.1.0",
)
def foo():
"""Foo"""
print("Yolooooooooo!")
@versionadded(
reason="Old function was inefficient",
version="0.1.0",
)
def new_foo():
print("Let's get serious!")
Rename function
from deprecated.sphinx import deprecated, versionchanged
@deprecated(
reason="Function changed name to :func:`utils.super_module.new_bar`",
version="0.1.0",
)
def foo():
return bar()
@versionchanged(
reason="Function changed with a better name",
version="0.1.0",
)
#### PUT HERE ANY PREVIOUS DEPRECATION DECORATORS OF OLD FUNCTION
def bar():
print("Hello I have a fancy name.")
Rename class
from deprecated.sphinx import deprecated, versionchanged
@versionchanged(
reason="Class changed with a better name",
version="0.2.0",
)
class B:
def __init__(self, a):
self.a = a
@deprecated(
reason="Class changed name to :func:`utils.super_module.B`",
version="0.2.0",
)
class A(B):
pass
Change function/class API
import warnings
from deprecated.sphinx import versionchanged
@versionchanged(reason="API change", version="0.2.0")
def super_print(*args):
match args:
case prefix, text:
_new_super_print(prefix, text)
case text:
warnings.warn("API changed!", FutureWarning)
_old_super_print(text)
def _old_super_print(text):
_new_super_print("Super", text)
def _new_super_print(prefix, text):
print(f"{prefix} {text}")
Change function/class behaviour
from deprecated.sphinx import deprecated, versionchanged
@versionchanged(reason="Behaviour has changed!", version="1.0")
def operation(a, b):
"""Future behaviour"""
return a + b
@deprecated(reason="Behaviour will change", version="0.2.3")
def operation(a, b):
"""Legacy behaviour"""
return a * b