SalishSeaNowcast Package Development

Continuous Integration

 Pytest with Coverage Status Codecov Testing Coverage Report CodeQL analysis

Documentation

 Documentation Status Sphinx linkcheck

Package

 Releases Python Version from PEP 621 TOML Issue Tracker

Meta

 Licensed under the Apache License, Version 2.0 Git on GitHub Pixi pre-commit The uncompromising Python code formatter Hatch project

The SalishSeaNowcast package is a collection of Python modules associated with running the SalishSeaCast ocean models in a daily nowcast/forecast mode. The package uses the NEMO_Nowcast framework to implement the SalishSeaCast Ocean Model Automation System.

Python Version

Python Version from PEP 621 TOML

The SalishSeaNowcast package is developed and tested using Python 3.13.

Getting the Code

Git on GitHub

Clone the code and documentation repository from GitHub with:

$ git clone git@github.com:SalishSeaCast/SalishSeaNowcast.git

Development Environment

Pixi

SalishSeaNowcast uses Pixi for package and environment management. If you don’t already have Pixi installed, please follow its installation instructions to do so.

Use pixi install command to download the package dependencies and link them into environments.

Most commands are executed using pixi run in the SalishSeaNowcast/ directory (or a sub-directory).

  • The default environment has the packages installed that are required to run the modules that have a command-line interface; e.g. pixi run python -m nowcast.workers.collect_weather --help

  • Other environments used by commands in the sections below have addition packages for running the test suite, building and link checking the documentation, etc.

  • If you are using an integrated development environment like VSCode or PyCharm where you need a Python interpreter to support coding assistance features, run development tasks, etc., use the interpreter in the dev environment. You can get its full path with pixi run -e dev which python

You can launch a sub-shell in one of the environments with a command like pixi shell -e dev. That is convenient if you are running a lot of commands because it removed the need to type pixi run -e dev before each of them. Use exit to leave the sub-shell.

SalishSeaNowcast depends on a collection of other Python packages developed by the SalishSeaCast project and friends:

Those packages are installed by the pixi install command.

To get detailed information about the environments, the packages installed in them, Pixi tasks that are defined for them, etc., use pixi info.

SalishSeaNowcast is installed in editable install mode in all of the environments that Pixi creates. That means that changes you make to the code are immediately reflected in the environments.

Coding Style

pre-commit The uncompromising Python code formatter

The SalishSeaNowcast package uses Git pre-commit hooks managed by pre-commit to maintain consistent code style and and other aspects of code, docs, and repo QA.

To install the pre-commit hooks in a newly cloned repo, run pre-commit install:

$ cd SalishSeaNowcast
$ pixi run -e dev pre-commit install

Note

You only need to install the hooks once immediately after you make a new clone of the SalishSeaNowcast repository and build your Development Environment.

Building the Documentation

Documentation Status

The documentation for the SalishSeaNowcast package is written in reStructuredText and converted to HTML using Sphinx. Creating a Development Environment as described above includes the installation of Sphinx. Building the documentation is driven by the docs/Makefile. To do a clean build of the documentation use:

$ cd SalishSeaNowcast
$ pixi run docs

The output looks something like:

 ✨ Pixi task (docs in docs): make clean html
 Removing everything under '_build'...
 Running Sphinx v9.1.0
 loading translations [en]... done
 making output directory... done
 loading intersphinx inventory 'python' from https://docs.python.org/3/objects.inv ...
 loading intersphinx inventory 'nemonowcast' from https://nemo-nowcast.readthedocs.io/en/latest/objects.inv ...
 loading intersphinx inventory 'salishseatools' from https://salishsea-meopar-tools.readthedocs.io/en/latest/objects.inv ...
 loading intersphinx inventory 'salishseasite' from https://salishsea-site.readthedocs.io/objects.inv ...
 loading intersphinx inventory 'salishseadocs' from https://salishsea-meopar-docs.readthedocs.io/en/latest/objects.inv ...
 loading intersphinx inventory 'salishseacmd' from https://salishseacmd.readthedocs.io/en/latest/objects.inv ...
 building [mo]: targets for 0 po files that are out of date
 writing output...
 building [html]: targets for 20 source files that are out of date
 updating environment: [new config] 20 added, 0 changed, 0 removed
 reading sources... [100%] workers
 looking for now-outdated files... none found
 pickling environment... done
 checking consistency... done
 preparing documents... done
 copying assets...
 copying static files...
 Writing evaluated template result to /media/doug/warehouse/MEOPAR/SalishSeaNowcast/docs/_build/html/_static/language_data.js
 Writing evaluated template result to /media/doug/warehouse/MEOPAR/SalishSeaNowcast/docs/_build/html/_static/documentation_options.js
 Writing evaluated template result to /media/doug/warehouse/MEOPAR/SalishSeaNowcast/docs/_build/html/_static/basic.css
 Writing evaluated template result to /media/doug/warehouse/MEOPAR/SalishSeaNowcast/docs/_build/html/_static/js/versions.js
 Writing evaluated template result to /media/doug/warehouse/MEOPAR/SalishSeaNowcast/docs/_build/html/_static/copybutton.js
 copying static files: done
 copying extra files...
 copying extra files: done
 copying assets: done
 writing output... [100%] workers
 generating indices... genindex py-modindex done
 highlighting module code... [100%] nowcast.workers.watch_ww3
 writing additional pages... search done
 copying images... [100%] ProcessFlow.png
 dumping search index in English (code: en)... done
 dumping object inventory... done
 build succeeded.

 The HTML pages are in _build/html.

The HTML rendering of the docs ends up in docs/_build/html/. You can open the index.html file in that directory tree in your browser to preview the results of the build.

If you have write access to the repository on GitHub, whenever you push changes to GitHub the documentation is automatically re-built and rendered at https://salishsea-nowcast.readthedocs.io/en/latest/.

Running the Unit Tests

The test suite for the SalishSeaNowcast package is in SalishSeaNowcast/tests/. The pytest tool is used for test parametrization and as the test runner for the suite.

Use:

$ cd SalishSeaNowcast/
$ pixi run -e test pytest

to run the test suite. The output looks something like:

 ================================ test session starts =================================
 platform linux -- Python 3.14.6, pytest-9.1.1, pluggy-1.6.0
 Using --randomly-seed=1914058026
 rootdir: /media/doug/warehouse/MEOPAR/SalishSeaNowcast
 configfile: pyproject.toml
 plugins: httpx-0.36.2, randomly-3.15.0, anyio-4.14.1, cov-7.1.0
 collected 2036 items

 tests/workers/test_collect_weather.py ................................................
 .....                                                                           [  2%]
 tests/workers/test_update_forecast_datasets.py .......................................
 ...............................                                                 [  6%]
 tests/workers/test_make_plots.py .....................................................
 ............                                                                    [  9%]
 tests/workers/test_download_wwatch3_results.py ..........                       [  9%]
 tests/workers/test_make_forcing_links.py .............................................
 .................                                                               [ 12%]
 tests/workers/test_ping_erddap.py .................................             [ 14%]
 tests/workers/test_collect_NeahBay_ssh.py ...................                   [ 15%]
 tests/workers/test_run_NEMO.py .......................................................
 .............................................................................   [ 21%]
 tests/workers/test_collect_river_data.py .............................          [ 23%]
 tests/test_analyze.py .................                                         [ 24%]
 tests/workers/test_make_CHS_currents_file.py ........................           [ 25%]
 tests/workers/test_run_ww3.py ........................................................
 .............                                                                   [ 28%]
 tests/workers/test_grib_to_netcdf.py .................................................
 ............                                                                    [ 31%]
 tests/workers/test_make_feeds.py ...................                            [ 32%]
 tests/workers/test_make_averaged_dataset.py ..................................
 ........                                                                        [ 34%]
 tests/test_next_workers.py ...........................................................
 ......................................................................................
 ........................................                                        [ 43%]
 ......................................................................................
 .......................................                                         [ 49%]
 tests/release_mgmt/test_tag_release.py .........                                [ 50%]
 tests/test_residuals.py ...                                                     [ 50%]
 tests/test_daily_river_flows.py ......................................          [ 52%]
 tests/workers/test_make_ww3_wind_file.py ........................               [ 53%]
 tests/workers/test_watch_NEMO.py .....................................................
 .........................                                                       [ 57%]
 tests/workers/test_upload_forcing.py .................................................
 .......................                                                         [ 60%]
 tests/workers/test_make_turbidity_file.py ....                                  [ 61%]
 tests/workers/test_watch_NEMO_hindcast.py ............................................
 .................                                                               [ 64%]
 tests/workers/test_download_weather.py ...............................................
 .........                                                                       [ 66%]
 tests/workers/test_get_onc_ctd.py ............                                  [ 67%]
 tests/workers/test_make_runoff_file.py ...............................................
 ...........................................................................     [ 73%]
 tests/workers/test_get_vfpa_hadcp.py ..............                             [ 74%]
 tests/workers/test_rotate_hindcast_logs.py ..........                           [ 74%]
 tests/workers/test_make_ssh_file.py .................                           [ 75%]
 tests/workers/test_get_onc_ferry.py ..................................          [ 77%]
 tests/workers/test_make_ww3_current_file.py ...............................     [ 78%]
 tests/workers/test_make_surface_current_tiles.py ...........................    [ 79%]
 tests/workers/test_watch_ww3.py ..................                              [ 80%]
 tests/workers/test_make_live_ocean_files.py .........                           [ 81%]
 tests/workers/test_archive_tarball.py ........................                  [ 82%]
 tests/workers/test_watch_NEMO_agrif.py ..................                       [ 83%]
 tests/workers/test_download_results.py ...............................................
 ...................                                                             [ 86%]
 tests/workers/test_launch_remote_worker.py ............                         [ 87%]
 tests/test_config.py ..............................                             [ 88%]
 tests/workers/test_download_live_ocean.py .........                             [ 89%]
 tests/workers/test_crop_gribs.py .............................................. [ 91%]
 tests/workers/test_run_NEMO_agrif.py ......................                     [ 92%]
 tests/workers/test_run_NEMO_hindcast.py ..............................................
 ......................................................................................
 ..........                                                                      [ 99%]
 tests/workers/test_split_results.py .............                               [100%]

 =============================== 2036 passed in 24.51s ================================

You can monitor what lines of code the test suite exercises using the coverage.py and pytest-cov tools with the command:

$ cd SalishSeaNowcast/
$ pixi run -e test pytest-cov

The test coverage report will be displayed below the test suite run output.

Alternatively, you can use

$ pixi run -e test pytest-cov-html

to produce an HTML report that you can view in your browser by opening SalishSeaNowcast/htmlcov/index.html.

Continuous Integration

GitHub Workflow Status

The SalishSeaNowcast package unit test suite is run and a coverage report is generated whenever changes are pushed to GitHub. The results are visible on the repo actions page, from the green checkmarks beside commits on the repo commits page, or from the green checkmark to the left of the “Latest commit” message on the repo code overview page . The testing coverage report is uploaded to codecov.io

The GitHub Actions workflow configuration that defines the continuous integration tasks is in the .github/workflows/pytest-coverage.yaml file.

Version Control Repository

Git on GitHub

The SalishSeaNowcast package code and documentation source files are available as a Git repository at https://github.com/SalishSeaCast/SalishSeaNowcast.

Issue Tracker

Issue Tracker

Development tasks, bug reports, and enhancement ideas are recorded and managed in the issue tracker at https://github.com/SalishSeaCast/SalishSeaNowcast/issues.

License

Licensed under the Apache License, Version 2.0

The SalishSeaCast NEMO model nowcast system code and documentation are copyright 2013 – present by the SalishSeaCast Project Contributors and The University of British Columbia.

They are licensed under the Apache License, Version 2.0. https://www.apache.org/licenses/LICENSE-2.0 Please see the LICENSE file for details of the license.

Release Process

Releases Hatch project

Releases are done at Doug’s discretion when significant pieces of development work have been completed.

The release process steps are:

  1. Use pixi run -e dev hatch version release to bump the version from .devn to the next release version identifier; e.g. 26.1.dev0 to 26.1

  2. Use pixi update to update the lock file to reflect the new version identifier

  3. Commit the version bump

  4. Create an annotated tag for the release with Git -> New Tag… in PyCharm or git tag -e -a vyy.n; e.g. git tag -e -a v26.1

  5. Push the version bump commit and tag to GitHub

  6. Use the GitHub web interface to create a release, editing the auto-generated release notes into sections:

### Features

### Bug Fixes

### Documentation

### Maintenance

### Dependency Updates

  1. Use the GitHub Issues -> Milestones web interface to edit the release milestone:

    • Change the Due date to the release date

    • Delete the “when it’s ready” comment in the Description

  2. Use the GitHub Issues -> Milestones web interface to create a milestone for the next release:

    • Set the Title to the next release version, prepended with a v; e.g. v26.2

    • Set the Due date to the end of the year of the next release

    • Set the Description to something like v26.2 release - when it's ready :-)

    • Create the next release milestone

  3. Review the open issues, especially any that are associated with the milestone for the just released version, and update their milestone.

  4. Close the milestone for the just released version.

  5. Use pixi run -e dev hatch version minor,dev to bump the version for the next development cycle, or use pixi run -e dev hatch version major,minor,dev for a year rollover version bump

  6. Use pixi update to update the lock file to reflect the new version identifier

  7. Commit the version bump

  8. Push the version bump commit to GitHub