Repository navigation
[Proposal] Refactor project structure #616
Description
After the CI build is merged (#614 #613), I believe the package structure needs to be addressed. Currently, the package structure obfuscates the build structure which makes contributing as a new developer is challenging. I would therefore propose to refactor the build structure to address these concerns. This would address point 3 raised here. The following structure would work:
/basemap
├── basemap/
├── data/
│ ├── basemap_data/
│ ├── basemap_hires/
├── pyproject.toml
├── build scripts
└── README.md
└── examples
To achieve this, we need to
- update the CI builds with the new structure
- Unify the
setup.pyandsetup.cfginpyproject.tomlfor a single point of entry to understand how the package is built - Move towards implicit namespaces
This restructuring would create a better user experience, allow for easier maintenance, consistent versioning, and simplify the build process.
Happy to hear your thoughts!
Activity
DWesl commentedon Mar 1, 2025
I don't know that a
pyproject.toml-only setup is possible quite yet, since setuptools is still working out how to specify C extension modules there, so at least that would still needsetup.py. Most of the rest ofsetup.cfgsupportspyproject.tomlnow.I also like the
srclayout to make it a bit harder to make certain kinds of mistakes in development, but CI will catch some of them and the build-from-sdist CI will catch morecvanelteren commentedon Mar 1, 2025
Could you elaborate on this?
molinav commentedon Mar 1, 2025
The
srclayout is a safeguard to ensure that the test phase is performed on the installedbasemapwheel and not on thebasemapfolder that you would have in the flat space. Otherwise, if your working directory is the package root, thebasemapfolder gets preference over the installed wheel when imported in the tests, and it could be that thebasemapfolder works but the wheel does not (i.e. because the wheels forgets to include something from the package tree), and you would not notice it.The build-from-sdist is a safeguard to ensure that the wheels can be properly built from the source distribution that is uploaded to PyPI. Building the wheels directly from the package tree does not check that the sdist contains everything needed to build the wheel, and this may result in broken packages for the people that make
pip install basemapwithout a precompiled wheel available (e.g. Cygwin). Indeed, this already occurred in the past (due to my lack of knowledge about it at the time), because the requirements file were not listed in theMANIFEST.infile and they were not included in the sdist.molinav commentedon Jun 13, 2025
Hi @cvanelteren. I am about to merge an issue branch that addresses your proposal partially.
I have converted again the root folder as the top folder for the
basemappackage. Thebasemap-dataandbasemap-data-hirespackages have been moved to adatafolder, and I have made the appropriate changes:datafrom thebasemappackage manifest.On the other side, the three packages will keep the skeleton based on
srcfolders, instead of the skeleton based on plain layout, because it helps with the maintenance of the project and avoids some subtle errors during development.For the data packages, I have removed the
setup.cfgfile, andsetup.pyis currently the single project configuration file. It can be considered at a later stage if we migrate everything to apyproject.tomlfile.For the
basemappackage, I have removed thesetup.cfgfile, but both thesetup.pyandpyproject.tomlfiles still need to stay, becausesetup.pystores logic needed to build the package.Add reference to issue #616 in CHANGELOG
molinav commentedon Jun 13, 2025
The changes are now merged into the
developbranch and will go into the 2.0.0 release that I will cut in the next couple of hours.