Repository
The git repos we are maintaining use multiple branches with a specific scope:
master
: for release code usable by the publicdev
: for the code being developed (between releases)n-xxxx
: all temporary branches called focusing on resolution of an issue number#n
with titlexxxx
(be it a feature ar bug)
The common master
and dev
branches are protected and
developers must not work directly on them.
When starting a new development, you should always span a new temporary
branch from the common dev
branch. We strongly recommend that each
new feature development or bug fix gets its own branch.
When you have checked extensively the modifications made on a feature
branch, you can then merge them on the common dev
branch. As a
matter of fact, always check extensively your changes before merging
them on any common branch. This merge should be made using a gitlab
merge request.
After each block of code implementing a feature that has been checked, you must, without exception, commit and push.
Commits
Your commits should be frequent and atomic (never bundle lots of
changes to multiple features inside a single commit). If you have made
a lot of changes, you can add file changes to the commit interactively
using git add -p
.
Do not underestimate the value of a clear-cut, well formatted commit message. We recommend the following guidelines to be used when writing a commit message: https://chris.beams.io/posts/git-commit/
For the format of the commit message, we demand the following:
First line (header): commit short message (again see https://chris.beams.io/posts/git-commit/)
Body: details about the commit (if needed), written as complete sentences
Last line (if applicable): references to the issue(s) related to this commit (ex:
See #1 #7
orFix #5
)
Changelog
A changelog file CHANGES
must be maintained, detailing the changes
brought about by a new revision. It is recommended you modify it along
your other commits to not miss anything. Each set of changes is
associated to a version number, which is associated to a
tag.
We are using the following format:
...
v{a}.{b}.{c+1}
--------------
- change1
- subchange1
...
- change2
...
v{a}.{b}.{c}
------------
(start of the changelog)
So each new version changes are appended above the changelog.
Between two releases, new changes are added to the top without a version number as section header (as version number still need to be determined). The addition as a version number as the top section header must be done upon releasing a new version.
Package requirements
It is mandatory that you keep an up-to-date file requirements.txt
at
the root of your project directory containing the package dependencies.
You can update it through:
$ pip freeze > requirements.txt
In this purpose, it is best to have set a python virtual environment to not gather requirements of other projects. If you need help setting that up, see section Python environment.
Makefile
For easier usability, we provide a Makefile for developers. It defines many commands and their options, the more useful ones being:
make lint
: check all linters + mypymake lint-apply
: apply isort and black formattingmake test
: perform unit testingmake coverage
: perform coverage tests on unit testingmake coverage-report
: print the coverage of each source filemake coverage-report-html
: show the coverage result in details in folderhtmlcov
make doc
: generate the documentation with sphinxmake notebook-test
: execute all notebooks in and from doc/notebooks/
Gitlab CI/CD pipeline
This repository has an associated pipeline for continuous integration. It consists of a succession of operations that are applied on each commit and merge-request pushed on the online repository.
This pipeline runs the following jobs on every change pushed:
Lint: test code readability and coherence (see linters)
black
flake8
isort
mypy
Test
coverage tests (run unit tests and compute coverage)
coverage html report (same as abobe but outputs the report in html detailed form)
Build
doc (check errors when generating the doc)
Integration tests
py{3.x} (one job per version of python >=3.8 to run unit tests in)
Backward (test retrocompatibility)
backward-major (test latest major release unit tests on new code)
backward-minor (test latest minor release unit tests on new code)
backward-patch (test latest patch release unit tests on new code)
Note
while it is a good practice to only push commits that are
guaranteed to pass those tests for every branch (by running those tests
locally beforehand), they are only required to pass for the common
dev
and master
branch.
In the case of a release (i.e a push on the master
branch), the
following jobs are appended:
Integration tests
install (test if the package can be installed from its source code)
notebooks (run all notebooks doc/notebooks/ and check they are working)
Build (build the distribution of the package)
Deploy (manually triggered: deploy new version of the package on PyPI (mcda)
Package structure
This package uses the following structure:
.
├── README.md # package readme in .md format
├── README.PYPI.md # PyPI package readme in .md format (deployed on PyPI)
├── CHANGES.txt # package changelog txt format
├── Dockerfile # development container dockerfile
├── Makefile # contains all helpers/utilities for developers
├── requirements.txt # up-to-date list of python packages and versions required (obtained with `pip freeze > requirements.txt`)
├── MANIFEST.in # indicates all non-source files to include when building package
├── pyproject.toml # Minimal configuration to build package distributed binaries
├── .gitlab-ci.yml # Continuous Integration script for gitlab pipeline
├── .readthedocs.yaml # Configuration for readthedocs integration
├── tox.ini # tox configuration file (used to test package vs other python versions)
├── setup.py # package installation setup
├── src # folder for python sources structured for packaging
│ .
│ └── mcda # package source code
│ ├── . # User API (externally usable features) package structure
│ └── internal # folder for package internal features
├── tests # folder containing python tests (unit or coverage)
│ .
├── doc # documentation folder
│ .
│ ├── notebooks # folder containing examples and tutorials notebooks
│ └── html # html documentation folder
├── scripts # folder with development useful scripts
│ .
├── .vscode # VS Code configuration (that we use as maintainers)
│ .
├── .devcontainer # VS Code development container configuration files
│ .
├── LICENSE # present package intent, license choice and authors (english)
├── LICENCE
│ ├── {complete_license_file}_EN.txt # complete license file in english
│ ├── {complete_license_file}_FR.txt # complete license file in french
│ ├── LICENCE.en # present package intent, license choice and authors (english)
│ └── LICENCE.fr # present package intent, license choice and authors (french)
│ .