Metadata-Version: 2.4
Name: lcfm-shapefiles
Version: 0.9.12
Summary: Add your description here
Author-email: Filip Schouwenaars <filip.schouwenaars@vito.be>, Lagaras Stelios <stelios.lagaras@vito.be>
Requires-Python: >=3.10
Requires-Dist: loguru
Requires-Dist: numpy
Requires-Dist: shapely<3
Provides-Extra: notebooks
Requires-Dist: jupyter; extra == 'notebooks'
Provides-Extra: scripts
Requires-Dist: matplotlib; extra == 'scripts'
Requires-Dist: satio[legacy]>=2.1.8; extra == 'scripts'
Requires-Dist: tqdm; extra == 'scripts'
Description-Content-Type: text/markdown

[![Auto-Sync LFS Files Github](https://github.com/VITO-RS-Vegetation/lcfm-shapefiles/actions/workflows/auto-sync-github.yml/badge.svg)](https://github.com/VITO-RS-Vegetation/lcfm-shapefiles/actions/workflows/auto-sync-github.yml)

# lcfm-shapefiles

Python wrapper around shapefiles used for the LCFM project.

## Install

```bash
uv add --index https://artifactory.vgt.vito.be/artifactory/api/pypi/python-packages/simple lcfm-shapefiles
```

## Use

```python
import lcfm_shapefiles import list_filepaths, get_filepath
import geopandas as gpd

# List out all available files; you can index this list to get a path
lcfm_shapefiles.list_filepaths()

# Get path a specific file
fp = lcfm_shapefiles.get_filepath("LCFM_1p_S2-tiles.fgb")

# Import fgb into a geopandas data frame
gdf = gpd.read_file(fp)
```

## Develop

```bash
# Build venv
uv sync

# Run tests
uv run pytest

# Format the code
uv run ruff format .
```

## CI/CD

A [Jenkins job](https://jenkins.vgt.vito.be/job/App/job/lcfm-shapefiles/job/master/) is configured, that:

- Builds the package
- Runs the tests
- Uploads a 'dev build' package to Artifactory, e.g. `0.2.3.dev20250305+6`
- Creates a promotion job.

When the build is successful, you should receive an email with a link to the promotion job.
If you don't get this email (e.g., when Jenkins can't infer your email from your commits), you can find the link to the promotion job in the build logs, or [here](https://jenkins.vgt.vito.be/job/App/job/promotion_jobs/job/lcfm-shapefiles/).
When you build this promotion job ("Build with Parameters" and then "Build"), it:

- Tags the commit based on the package version
- Uploads the 'tag build' package to Artifactory, e.g. `0.2.3`.

This CI/CD uses the [general Python pipeline](https://confluence.vito.be/display/EP/Python+pipeline) and was created through the [Jenkins job builder](https://confluence.vito.be/display/EP/Creating+Jenkins+jobs). Check with the Software Operations team for more details on its configuration or help with build issues.

### Steps to to deploy a new or update existing shapefiles

- Add or update shapefiles in `src/lcfm_shapefiles/data`.
- Update tests accordingly.
- Bump the package version in `pyproject.toml`.
- Commit and push the changes to `master`.
- If the build passes, trigger the promotion job.

A new version of the `lcfm-shapefiles` package should now be on Artifactory.

### Use Git LFS storage
We are using the Github Large File Storage [about](https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-git-large-file-storage) to store fgb files
This works alongside with the principle that big files should not be stored in the repository, and a pointer is stored in their place. The actual files are stored somewhere from LFS and are synchronized/cloned from our master branch in the vm at: `/vitodata/vegteam/auxdata/LCFM/shapefiles-lfs`

```bash
sudo dnf install -y git-lfs
git lfs install
```

If you need to pull the actual fgb files try:
```bash
git lfs pull
```

## MD5 and Shapefile limits
To ensure validity of downloaded assets, multi-part upload limit is set to 5GB aligning with the Cloudferro's suggested max size [recommended limit](https://cloudferro.com/cloud/public-cloud/storage/object-storage/)

## Auto-Syncing Shapefiles to Terrascope
When triggered, the script `/vitodata/vegteam/auxdata/LCFM/scripts/sync-shapefiles-terrascope.sh` performs a syncing from the master branch
As Terrascope network is not exposed publicly and github event-hooks can not be used, the trigger is a weekly defined cronjob:

### Cronjob
Cronjob is added with:
```bash
crontab -e
```
and then usin VIM editor add:
```txt
0 0 * * 0 /vitodata/vegteam/auxdata/LCFM/scripts/sync-shapefiles-terrascope.sh >> /vitodata/vegteam/auxdata/LCFM/logs/cron.log 2>&1
```
confirm that cronjob is added
```bash
crontab -l
```
 