Repository ========== The git repos we are maintaining use multiple branches with a specific scope: - ``master``: for release code usable by the public - ``dev``: for the code being developed (between releases) - ``n-xxxx``: all temporary branches called focusing on resolution of an issue number ``#n`` with title ``xxxx`` (be it a feature ar bug) **The common** ``master`` **and** ``dev`` **branches are protected and developers must not work directly on them.** When starting a new development, you should always span a new temporary branch from the common ``dev`` branch. **We strongly recommend that each new feature development or bug fix gets its own branch.** When you have checked extensively the modifications made on a feature branch, you can then merge them on the common ``dev`` branch. As a matter of fact, always check extensively your changes before merging them on any common branch. This merge should be made using a gitlab merge request. After each block of code implementing a feature that has been checked, you **must**, **without exception**, commit and push. Commits ------- Your commits should be frequent and atomic (**never bundle lots of changes to multiple features inside a single commit**). If you have made a lot of changes, you can add file changes to the commit interactively using ``git add -p``. **Do not underestimate the value of a clear-cut, well formatted commit message.** We recommend the following guidelines to be used when writing a commit message: https://chris.beams.io/posts/git-commit/ For the format of the commit message, we demand the following: - First line (header): commit short message (again see https://chris.beams.io/posts/git-commit/) - Body: details about the commit (if needed), written as complete sentences - Last line (if applicable): references to the issue(s) related to this commit (ex: ``See #1 #7`` or ``Fix #5``) Tags ---- Tags are used (among other things) to indicates the commit number of a specific version of the repo. This is important to put in place once the repo is made available to the public so users and developers alike can track versions. We use three numbers identifying a version: ``a.b.c`` - ``a``: indicates major revisions (lot of changes perhaps with no backward compatibility) - ``b``: indicates minor revisions (new feature introduced or a bunch of smaller ones) - ``c``: indicates patch or bug-fix Those version numbers should be at least used for the ``master`` branch, and can indicate releases. The tags indicating versions should be formatted: ``v{a}.{b}.{c}`` (for example: ``v1.0.1``) You can get the commits associated with each tag, which gives you the version number of these commits. But you can also get the version number associated with any commit in the history, by calling the following command (having checkout the commit to version): .. code:: console $ git describe --tags --match "v[0-9]*" This command will generate a version number with the following format: ``LAST_TAG_NAME-NB_COMMITS-COMMIT-CHECKSUM`` with ``LAST_TAG_NAME`` the closest tag in the commit history, ``COMMIT_CHECKSUM`` the checksum of the current commit and ``NB_COMMIT`` the number of commits separating the commit from the tag. For example in our case, with a commit checksum “gfface8d”, 3 commits after commit tagged with version “v1.2.0”, we get: ``v1.2.0-3-gfface8d`` Changelog --------- A changelog file ``CHANGES`` must be maintained, detailing the changes brought about by a new revision. It is recommended you modify it along your other commits to not miss anything. Each set of changes is associated to a version number, which is associated to a `tag <#tags>`__. We are using the following format: :: ... v{a}.{b}.{c+1} -------------- - change1 - subchange1 ... - change2 ... v{a}.{b}.{c} ------------ (start of the changelog) So each new version changes are appended above the changelog. Between two releases, new changes are added to the top without a version number as section header (as version number still need to be determined). The addition as a version number as the top section header must be done upon releasing a new version. Package requirements -------------------- It is mandatory that you keep an up-to-date file ``requirements.txt`` at the root of your project directory containing the package dependencies. You can update it through: .. code:: console $ pip freeze > requirements.txt In this purpose, it is best to have set a python virtual environment to not gather requirements of other projects. If you need help setting that up, see section :ref:`Python environment `. Makefile -------- For easier usability, we provide a `Makefile `__ for developers. It defines many commands and their options, the more useful ones being: - ``make lint``: check all linters + mypy - ``make lint-apply``: apply isort and black formatting - ``make test``: perform unit testing - ``make coverage``: perform coverage tests on unit testing - ``make coverage-report``: print the coverage of each source file - ``make coverage-report-html``: show the coverage result in details in folder ``htmlcov`` - ``make doc``: generate the documentation with sphinx - ``make notebook-test``: execute all notebooks in and from `doc/notebooks/ `__ Gitlab CI/CD pipeline --------------------- This repository has an associated pipeline for continuous integration. It consists of a succession of operations that are applied on each commit and merge-request pushed on the online repository. This pipeline runs the following jobs on every change pushed: - Lint: test code readability and coherence (see :ref:`linters `) - black - flake8 - isort - mypy - Test - coverage tests (run unit tests, compute coverage and create html report) - Build - doc (check errors when generating the doc) - Integration tests - py{3.x} (one job per version of python >=3.9 to run unit tests in) - Backward (test retrocompatibility) - backward-major (test latest major release unit tests on new code) - backward-minor (test latest minor release unit tests on new code) - backward-patch (test latest patch release unit tests on new code) .. note:: while it is a good practice to only push commits that are guaranteed to pass those tests for every branch (by running those tests locally beforehand), they are only required to pass for the common ``dev`` and ``master`` branch. In the case of a release (i.e a push on the ``master`` branch), the following jobs are appended: - Integration tests - notebooks (run all notebooks `doc/notebooks/ `__ and check they are working) - Build (build the distribution of the package) - Deploy - deploy new version of the package to PyPI (manually triggered) (`mcda `__) - tag the release commit with version number `vX.Y.Z` - create the release on Gitlab (`Gitlab `__) Package structure ----------------- This package uses the following structure: :: . ├── README.md # package readme in .md format ├── README.PYPI.md # PyPI package readme in .md format (deployed on PyPI) ├── CHANGES.txt # package changelog txt format ├── Dockerfile # development container dockerfile ├── Makefile # contains all helpers/utilities for developers ├── requirements.txt # up-to-date list of python packages and versions required (obtained with `pip freeze > requirements.txt`) ├── MANIFEST.in # indicates all non-source files to include when building package ├── pyproject.toml # Minimal configuration to build package distributed binaries ├── .gitlab-ci.yml # Continuous Integration script for gitlab pipeline ├── .readthedocs.yaml # Configuration for readthedocs integration ├── tox.ini # tox configuration file (used to test package vs other python versions) ├── setup.py # package installation setup ├── src # folder for python sources structured for packaging │ . │ └── mcda # package source code │ ├── . # User API (externally usable features) package structure │ └── internal # folder for package internal features ├── tests # folder containing python tests (unit or coverage) │ . ├── doc # documentation folder │ . │ ├── notebooks # folder containing examples and tutorials notebooks │ └── html # html documentation folder ├── scripts # folder with development useful scripts │ . ├── .vscode # VS Code configuration (that we use as maintainers) │ . ├── .devcontainer # VS Code development container configuration files │ . ├── LICENSE # present package intent, license choice and authors (english) ├── LICENCE │ ├── {complete_license_file}_EN.txt # complete license file in english │ ├── {complete_license_file}_FR.txt # complete license file in french │ ├── LICENCE.en # present package intent, license choice and authors (english) │ └── LICENCE.fr # present package intent, license choice and authors (french) │ .