FLOWFINDER: Watershed Delineation Research & Benchmark Framework

🎯 Research Questions

🔬 Research Context

Current State of Watershed Delineation

Research Gaps We're Addressing

📋 Prerequisites

🚀 Quick Start

1. Installation

# Clone the repository
git clone <repository-url>
cd flowfinder

# Install dependencies
pip install -e .[dev]

# Install FLOWFINDER
pip install flowfinder

# Copy environment template (if it exists)
cp .env.example .env || echo "Create .env file with your data paths"
# Edit .env with your data paths and configuration

2. Configuration Setup

# Configuration structure is already set up:

# Environment-specific configurations
config/environments/development.yaml    # Local dev (10 basins)
config/environments/testing.yaml        # CI/testing (50 basins)
config/environments/production.yaml     # Full-scale (500+ basins)

# Tool-specific configurations
config/tools/flowfinder.yaml            # FLOWFINDER settings
config/tools/taudem.yaml               # TauDEM MPI settings
config/tools/grass.yaml                # GRASS r.watershed settings
config/tools/whitebox.yaml             # WhiteboxTools settings

3. Data Preparation

data/
├── huc12_mountain_west.shp    # HUC12 boundaries for Mountain West
├── nhd_hr_catchments.shp      # NHD+ HR catchment polygons
├── nhd_flowlines.shp          # NHD+ HR flowlines
└── dem_10m.tif               # 10m DEM mosaic or tiles

4. Run Single-Tool Benchmark

# Step 1: Generate stratified basin sample
python scripts/basin_sampler.py --config config/basin_sampler_config.yaml

# Step 2: Extract truth polygons
python scripts/truth_extractor.py --config config/truth_extractor_config.yaml

# Step 3: Run FLOWFINDER benchmark
python scripts/benchmark_runner.py \
    --sample basin_sample.csv \
    --truth truth_polygons.gpkg \
    --config config/benchmark_config.yaml \
    --outdir results/

5. Run Multi-Tool Comparison (Experimental)

# Using the watershed experiment runner
python scripts/watershed_experiment_runner.py \
    --single --lat 40.0 --lon -105.5 --name "test_run" \
    --outdir results/multi_tool/

📁 Project Structure

├── README.md                    # Project overview + setup
├── requirements.txt             # Python dependencies
├── pyproject.toml              # Modern Python project config
├── .env.example                # Environment template
├── .gitignore                  # Standard Python gitignore
│
├── config/                     # Hierarchical configuration system
│   ├── base.yaml              # Foundation configurations
│   ├── configuration_manager.py # Configuration inheritance system
│   ├── schema.json            # JSON Schema validation
│   ├── environments/           # Environment-specific settings
│   │   ├── development.yaml   # Local development (10 basins)
│   │   ├── testing.yaml       # CI/testing (50 basins)
│   │   └── production.yaml    # Full-scale (500+ basins)
│   └── tools/                  # Tool-specific configurations
│       ├── flowfinder.yaml    # FLOWFINDER settings
│       ├── taudem.yaml        # TauDEM MPI settings
│       ├── grass.yaml         # GRASS r.watershed settings
│       └── whitebox.yaml      # WhiteboxTools settings
│
├── scripts/                    # Core benchmark scripts
│   ├── basin_sampler.py       # Stratified basin sampling
│   ├── truth_extractor.py     # Truth polygon extraction
│   ├── benchmark_runner.py    # FLOWFINDER accuracy testing
│   ├── watershed_experiment_runner.py # Multi-tool comparison
│   └── validation_tools.py    # Validation utilities
│
├── data/                       # Input datasets (gitignored)
├── results/                    # Output directory (gitignored)
├── tests/                      # Unit tests
├── docs/                       # Research and technical documentation
│   ├── strategic_analysis_implementation_roadmap_v2.md # Research roadmap
│   ├── multi_tool_integration_strategy.md # Integration approach
│   ├── strategic_analysis_assessment.md # Research evaluation
│   ├── immediate_next_steps.md # Implementation priorities
│   ├── configuration_architecture.md # Configuration system design
│   ├── multi_tool_benchmark_architecture.md # Framework design
│   └── test_coverage/          # Test coverage documentation
│
└── notebooks/                  # Jupyter exploration
    └── benchmark_analysis.ipynb

🔧 Configuration Architecture

Configuration Hierarchy

Base Configurations → Environment → Tool → Local Overrides

Example Configuration Composition

# Development FLOWFINDER experiment
inherits:
  - "base/regions.yaml#mountain_west_minimal"
  - "base/quality_standards.yaml#development_grade"
  - "environments/development.yaml"
  - "tools/flowfinder/base.yaml"
  - "experiments/accuracy_comparison.yaml"

overrides:
  basin_sampling:
    n_per_stratum: 1  # Minimal for dev
  benchmark:
    timeout_seconds: 30  # Quick timeout

Tool Adapter Interface

class ToolAdapter(ABC):
    @abstractmethod
    def delineate_watershed(self, pour_point: Point, dem_path: str) -> Tuple[Polygon, Dict]:
        """Delineate watershed and return polygon + performance metrics"""
        pass

    @abstractmethod
    def is_available(self) -> bool:
        """Check if tool is available on system"""
        pass

📊 Research Outputs

Single-Tool Benchmark

Multi-Tool Comparison (Experimental)

🎯 Research Metrics

Technical Validation

Research Impact Goals

Community Engagement Goals

🧪 Testing

# Run unit tests
python -m pytest tests/

# Test configuration system
python test_configuration_system.py

# Test multi-tool integration
python test_integration.py

# Run with coverage
python -m pytest tests/ --cov=scripts --cov-report=html

📈 Analysis

# Start Jupyter
jupyter lab notebooks/

# Open benchmark_analysis.ipynb for interactive exploration

🎯 Research Roadmap

Phase 1: Foundation - IN PROGRESS

Phase 2: Tool Integration - PLANNED

📚 Documentation

Research Documents

Technical Documents

🤝 Contributing

📄 License

🙏 Acknowledgments

📞 Support