salem-import/README.md

# Salem Oregon Address Import

See [https://wiki.openstreetmap.org/wiki/Salem_Oregon_Address_Import](https://wiki.openstreetmap.org/wiki/Salem_Oregon_Address_Import)

## Data Source

- Salem GIS department's FTP server
  - Also https://data.cityofsalem.net/datasets/salem::primary-address/explore
- No copyright, public domain

<img src="Screenshot_2023-10-25_213128.png"/>

## Field Mapping

- ADD_NUM -> addr:housenumber (int)
- formatstreet(FULL_ST_NAME) -> addr:street (text)
- SUB_VAL -> addr:unit (text)
- title(CITY) -> addr:city (text)
- ZIP -> addr:postcode (text)
- addr:state 'OR' (manually added)

## Processing

- Get `Primary_Addresses.geojson` from the City GIS server linked above.
- Open the file in QGIS and open the layer attribute table
- Use the field calculator and the below script to add new virtual fields per the mapping above. (You may have to hit Save and Load Functions again to refresh QGIS's memory)
- Export the layer to geojson with "EPSG:4326 - WGS 84" projection. Deselect all fields besides those above.
- Save as `processed.geojson`

## Import steps

- Load processed.geojson into JOSM
- Zoom in to the area you wish to work on and Download OSM data to a New Layer
- In the geojson layer, select one neighborhood or city block worth of addresses
- Click Edit > Merge Selection
- Delete the selected items you just merged (so you don't try merging them again later)
- Switch to Data Layer 1 and run the JOSM Validator
  - An easy way to only validate changes is to press the Upload button, but cancel before actually uploading.
  - We don't need to worry about validation errors that don't involve our changes.
- Fix all duplicate housenumber warnings and nearby street not found warnings
  - An easy way to auto-fix all duplicate housenumbers is to select all duplicates, Search within the selection for `new`, and delete.
  - Pay special attention to errors like "East Street Northeast" -> "E Street Northeast" which can be mass-corrected.
- Search for all `new "addr:housenumber" = "0"` elements and delete them.
- Click Upload, verify there are no further warnings or errors in the changeset
  - Make sure there are no erroneous Relations or other unwanted objects about to be uploaded.
- Upload with this changeset comment:

```
comment=Addresses near Salem Oregon #salemimport
import=yes
website=https://wiki.openstreetmap.org/wiki/Salem_Oregon_Address_Import
source=City of Salem GIS
source:url=https://data.cityofsalem.net/datasets/salem::primary-address/explore
```

- Review imported data in Achavi or Osmcha to ensure it looks proper.

## QGIS Processing script

```
import qgis.core
import qgis.gui
import re

#
# This will keep street names like SR 574A as SR 574A however
# will lowercase other number-digit suffixes with <2 or >4 numbers
# or >1 suffix-letters, like 12th Street or 243rd Ave.
#

@qgsfunction(args='auto', group='Custom', referenced_columns=[])
def getstreetfromaddress(value1, feature, parent):
    parts = value1.split()
    parts.pop(0) # Ignore the first bit (i.e. "123" in "123 N MAIN ST")
    parts = map(formatstreetname, parts)
    return " ".join(parts)

@qgsfunction(args='auto', group='Custom', referenced_columns=[])
def formatstreet(value1, feature, parent):
    parts = value1.split()
    # Handle the special case of a street name starting with "ST"
    # which is almost always "Saint __" and not "Street __"
    if parts[0] == "ST":
        parts[0] = "Saint"
    parts = map(formatstreetname, parts)
    return " ".join(parts)

# Internal function
def formatstreetname(name):
    # Acryonyms
    if name == "CR":
        return "County Road"
    if name == "SR":
        return "SR" # State Route
    if name == "NFS":
        return "NFS" # National Forest Service?
    if name == "US":
        return "US"
    # Directions
    if name == "N":
        return "North"
    if name == "NE":
        return "Northeast"
    if name == "E":
        return "East"
    if name == "SE":
        return "Southeast"
    if name == "S":
        return "South"
    if name == "SW":
        return "Southwest"
    if name == "W":
        return "West"
    if name == "NW":
        return "Northwest"
    # Suffixes
    if name == "AV":
        return "Avenue"
    if name == "AVE":
        return "Avenue"
    if name == "BLVD":
        return "Boulevard"
    if name == "BND":
        return "Bend"
    if name == "CIR":
        return "Circle"
    if name == "CR":
        return "Circle"
    if name == "CT":
        return "Court"
    if name == "DR":
        return "Drive"
    if name == "FLDS":
        return "Fields"
    if name == "GRV":
        return "Grove"
    if name == "HL":
        return "Hill"
    if name == "HOLW":
        return "Hollow"
    if name == "HW":
        return "Highway"
    if name == "HWY":
        return "Highway"
    if name == "LN":
        return "Lane"
    if name == "LOOP":
        return "Loop"
    if name == "LP":
        return "Loop"
    if name == "PATH":
        return "Path"
    if name == "PL":
        return "Place"
    if name == "RD":
        return "Road"
    if name == "RUN":
        return "Run"
    if name == "SQ":
        return "Square"
    if name == "ST":
        return "Street"
    if name == "TER":
        return "Terrace"
    if name == "TRL":
        return "Trail"
    if name == "VW":
        return "View"
    if name == "WAY":
        return "Way"
    if name == "WY":
        return "Way"
    if name == "XING":
        return "Crossing"
    if re.match('^[0-9]{2,4}[A-Za-z]$', name) != None:
        return name

    return name.capitalize()
```