salem-import/README.md

# Salem Oregon Address Import

See [https://wiki.openstreetmap.org/wiki/Salem_Oregon_Address_Import](https://wiki.openstreetmap.org/wiki/Salem_Oregon_Address_Import)

## Data Source

- Salem GIS department's FTP server
  - Also https://data.cityofsalem.net/datasets/salem::primary-address/explore
- No copyright, public domain

<img src="Screenshot_2023-10-25_213128.png"/>

## Field Mapping

- ADD_NUM -> addr:housenumber
- FULL_ST_NA -> addr:street via getformattedstreetname(FULL_ST_NA)
- SUB_VAL -> addr:unit
- CITY -> addr:city via title(CITY)
- ZIP -> addr:postcode

- manually add addr:state "OR"

## Processing

- Use the below script in QGIS to do virtual field calculations.
- Export the layer to geojson with WGS84 projection.
- Save as `processed.geojson`
- Use VIM or similar to find-replace "ADD_NUM" to "addr:housenumber" etc.

## Import steps

- Load processed.geojson into JOSM
- Zoom in to the area you wish to work on and Download OSM data to a New Layer
- In the geojson layer, select one neighborhood or city block worth of addresses
- Click Edit > Merge Selection
- Switch to Data Layer 1 and run the JOSM Validator
- Fix all duplicate housenumber warnings and nearby street not found warnings
- Click Upload, verify there are no further warnings or errors in the changeset
  - Make sure there are no erroneous Relations or other unwanted objects about to be uploaded.
- Upload with this changeset comment:

```
comment=Addresses near Salem Oregon #salemimport
import=yes
website=https://wiki.openstreetmap.org/wiki/Salem_Oregon_Address_Import
source=City of Salem GIS
source:url=https://data.cityofsalem.net/datasets/salem::primary-address/explore
```

- Review imported data in Achavi or Osmcha to ensure it looks proper.

## QGIS Processing script

```
from qgis.core import *
from qgis.gui import *
import re

@qgsfunction(args='auto', group='Custom', referenced_columns=[])
def getformattedstreetname(value1, feature, parent):
    parts = value1.split()
    parts = map(formatstreetname, parts)
    return " ".join(parts)

@qgsfunction(args='auto', group='Custom', referenced_columns=[])
def getformattedstreetnamefromaddress(value1, feature, parent):
    parts = value1.split()
    parts.pop(0) # Ignore the first bit (i.e. "123" in "123 N MAIN ST")
    parts = map(formatstreetname, parts)
    return " ".join(parts)

def formatstreetname(name):
    # Specific suffixes like "123th" we have lower
    if re.search("[0-9]+TH", name):
        return name.capitalize()
    if re.search("[0-9]+ND", name):
        return name.capitalize()
    if re.search("[0-9]+ST", name):
        return name.capitalize()
    if re.search("[0-9]+RD", name):
        return name.capitalize()
    # Weird names like 123D we keep upper
    if re.search("[0-9]+[A-Z]+", name):
        return name
    # Prefixes we want to keep uppercase
    if name == "US":
        return "US"
    if name == "SR":
        return "SR"
    if name == "CR":
        return "CR"
    if name == "C":
        return "C"
    # Directions
    if name == "N":
        return "North"
    if name == "NE":
        return "Northeast"
    if name == "E":
        return "East"
    if name == "SE":
        return "Southeast"
    if name == "S":
        return "South"
    if name == "SW":
        return "Southwest"
    if name == "W":
        return "West"
    if name == "NW":
        return "Northwest"
    # Suffixes
    if name == "AV":
        return "Avenue"
    if name == "AVE":
        return "Avenue"
    if name == "BLVD":
        return "Boulevard"
    if name == "BND":
        return "Bend"
    if name == "CIR":
        return "Circle"
    if name == "CT":
        return "Court"
    if name == "DR":
        return "Drive"
    if name == "FLDS":
        return "Fields"
    if name == "GRV":
        return "Grove"
    if name == "HOLW":
        return "Hollow"
    if name == "HW":
        return "Highway"
    if name == "HWY":
        return "Highway"
    if name == "LN":
        return "Lane"
    if name == "LP":
        return "Loop"
    if name == "LOOP":
        return "Loop"
    if name == "PATH":
        return "Path"
    if name == "PL":
        return "Place"
    if name == "RD":
        return "Road"
    if name == "RUN":
        return "Run"
    if name == "ST":
        return "Street"
    if name == "TER":
        return "Terrace"
    if name == "TL":
        return "Trail"
    if name == "TRL":
        return "Trail"
    if name == "VW":
        return "View"
    if name == "WAY":
        return "Way"
    if name == "XING":
        return "Crossing"
    # Irish names
    if name == "MCCRAY":
        return "McCray"
    if name == "MCKOWN":
        return "McKown"
    return name.capitalize()
```