Download county data — fetches the county dataset (or upload a shapefile ZIP for paths)
Download OSM data — queries Overpass for the relevant features in the county
Convert to GeoJSON — converts the county data to OSM-tagged GeoJSON
Diff vs OSM — compares county data against OSM; produces added (green) and removed (red) GeoJSON

Open the result in the Map Viewer. Green = features in county data missing from OSM (candidates to add). Red = features in OSM not in county data. Hide the OSM (gray) and county (purple) layers to focus on the diff. Click a feature to inspect its properties and accept or exclude it. Open accepted features in JOSM for final review before uploading.

Roads

Queries Overpass for all highway types: motorway, trunk, primary, secondary, tertiary, residential, unclassified, and their link variants, plus service and track.

Addresses

Run roads first — having street geometries in JOSM reduces address validation errors during upload. Queries Overpass for all addr:* nodes and ways. Easiest to also hide removed (red) and focus only on added (green) addresses missing from OSM.

Paths

Only Sumter currently has paths available. Queries Overpass for foot, bicycle, and multi-use paths.

Docker Quick Start

Using Docker Compose (Recommended)

Build and start the container:

docker-compose up -d --build

Note: The --build flag ensures the image is rebuilt with the latest code changes.

Access the web interface:

Main interface: http://localhost:5000
Map viewer: http://localhost:5000/map

Stop the container:

docker-compose down

Using Docker Directly

Build the image:

docker build -t osm-import-tools .

Run the container:

docker run -d \
  -p 5000:5000 \
  -v "$(pwd)/data:/data" \
  --name osm-import-tools \
  osm-import-tools

View logs:

docker logs -f osm-import-tools

Stop the container:

docker stop osm-import-tools
docker rm osm-import-tools

Deploying with Portainer

Just push to git and set up a pull from git; Portainer can build and run the project directly.

Uploading to a registry

docker build -t osm-import-tools .
docker tag osm-import-tools your-registry.example.com/you/osm-import-tools
docker push your-registry.example.com/you/osm-import-tools

Features

Main Dashboard (/)

Run data processing scripts per county (Roads, Paths, Addresses)
View real-time script output with colored status indicators
Live CPU/memory/disk usage charts during execution
Cancel running jobs
Quick-access links to processed GeoJSON files

Map Viewer (/map)

Interactive Leaflet map for comparing GeoJSON layers
Layer controls with drag-and-drop reordering
Filter by added (green) / removed (red) features
Hide service, track, and unclassified roads
Click features to inspect properties
Accept/reject diff features and save results
Multi-select mode (or Shift+drag) for bulk operations

Counties (/counties)

Define counties with ID, name, state, color, and download URLs
Download/upload the counties.yml config file

Street Exceptions (/exceptions)

Manage street name corrections (e.g. "D Angelo" → "D'Angelo")
Download/upload the exceptions.yml config file

Exclusions (/exclusions)

Define field/value pairs to exclude from diff output
Exclude button on map popups for quick additions
Download/upload the exclusions.yml config file

Authentication

Login required for all write operations
Admin user configured via ADMIN_PASSWORD environment variable

Volume Mounts

The Docker container mounts a single data directory:

./data → /data - All data files, organized by county

Inside /data, the structure is:

/data/lake/ - Lake County data
/data/sumter/ - Sumter County data
/data/counties.yml - County definitions
/data/exceptions.yml - Street name corrections
/data/exclusions.yml - Exclusion rules
/data/users.yml - User credentials

All changes are persisted on the host in the local ./data folder.

API Endpoints

Auth

GET /api/auth-status - Check authentication status

Script Execution

POST /api/run-script - Execute a processing script
GET /api/job-status/<job_id> - Get job status and logs
POST /api/cancel-job - Cancel a running job

File Operations

GET /api/list-files - List available GeoJSON files
POST /api/upload-file - Upload shapefile ZIP
POST /api/clear-data - Delete all county data files
GET /data/<path> - Serve data files

Configuration

GET /api/config/<filename> - Download a YAML config file
POST /api/config/<filename> - Upload a YAML config file
GET/POST /api/counties - Get or save county configuration
GET/POST /api/exceptions - Get or save street exceptions
GET/POST /api/exclusions - Get or save exclusions
POST /api/exclusions/add - Append new exclusions (skips duplicates)

System

GET /api/system-stats - CPU, memory, and disk usage

Troubleshooting

Port already in use

If port 5000 is already in use, edit docker-compose.yml:

ports:
  - "8080:5000"  # Change 8080 to any available port

Permission issues

Ensure the data directory has proper permissions:

mkdir -p data
chmod -R 755 data

View container logs

docker-compose logs -f

Rebuild after code changes

docker-compose down
docker-compose build --no-cache
docker-compose up -d

TODO

Validate whether a similar address is within 50 feet (same number?)
If addr:unit is like "Apartment 123" or "Apartment 123;Apartment 124;Apartment 125" remove the word Apartment so it's just "123" or "123;124;125"
If an item gets through processing and is "" it should be cleared
If a value is a range like "123 - 456" assume it's the same as "123-456" for matching purposes
Unit numbers with semicolons are still getting detected as different despite being identical, like: addr:city=Wildwood addr:housenumber=8655 addr:postcode=34785 addr:state=FL addr:street=Easterly Lane addr:unit=108;208;308
Some items appear duplicate in the data itself for unknown reasons: addr:city=Bushnell addr:housenumber=5740 addr:postcode=33513 addr:state=FL addr:street=Hideaway Circle
Sr is SR (State Route)