Lake county updates

README.md

@@ -4,21 +4,26 @@ See [https://wiki.openstreetmap.org/wiki/The_Villages_Road_and_Address_Import](h
 
 ## Instructions
 
+* Always do roads first, addresses second, so new subdivisions don't throw address validation errors.
 * Open the original data in QGIS
 * Format OSM fields with QGIS functions to have proper capitalization and full spellings without extraneous whitespace, based on original fields. For example OSM uses names like North Main Street, not N MAIN ST. All fields are of the QGIS type "text" even if they're numbers.
   * You can use the Attribute Table's Field Calculator for this; you can copy-paste the `qgis-functions.py` file into the Function Editor and then use the Expression tab to create new, formatted virtual fields. Don't worry if the field name limit is too short, it can be fixed in JOSM.
-* For addresses:
+
-  * The Addresses shapefile is recorded in the ESRI:102659 CRS, you may need to convert or reproject to/from.
+### For Sumter County:
-  * `ADD_NUM` becomes the virtual `addr:housenumber` (or `addr:house` temporarily, avoiding addr:house which is a real tag)
+
-  * `UNIT` becomes the virtual `addr:unit` (sometimes the LOT key is used for multiple units in a range, but mostly it's unrelated lot IDs and not useful)
-  * `SADD` becomes the virtual `addr:street` (or `addr:stree` temporarily) via the `getformattedstreetnamefromaddress("SADD")` custom expression
-  * `POST_COMM` becomes the virual `addr:city` via the `title("POST_COMM")` expression (we care about postal community addresses not what municipality a place might be governed by)
-  * `POST_CODE` becomes `addr:postcode` (or `addr:postc` temporarily)
 * For roads:
-  * `NAME` becomes the virtual `name` via the `getformattedstreetname("NAME")`
+  * `NAME` becomes the virtual `name` via the `title(formatstreet("NAME"))`
-  * `SpeedLimit` becomes the virtual `maxspeed` via `"SpeedLimit" mph`
+  * `SpeedLimit` becomes the virtual `maxspeed` via `concat("SpeedLimit",' mph')`
   * `highway=residential` or similar added manually in JOSM
   * `surface=asphalt` added manually in JOSM
+* For addresses:
+  * The Addresses shapefile is recorded in the ESRI:102659 CRS, you may need to convert or reproject to/from the default EPSG:4326 - WGS 84 CRS that OSM uses.
+  * `ADD_NUM` becomes the virtual `addr:housenumber` (or `addr:house` temporarily, avoiding addr:house which is a real tag) as an integer
+  * `UNIT` becomes the virtual `addr:unit` (sometimes the LOT key is used for multiple units in a range, but mostly it's unrelated lot IDs and not useful) as a string
+  * `SADD` becomes the virtual `addr:street` (or `addr:stree` temporarily) via the `getformattedstreetnamefromaddress("SADD")` custom expression as a string
+  * `POST_COMM` becomes the virual `addr:city` via the `title("POST_COMM")` expression (we care about postal community addresses not what municipality a place might be governed by) as a string
+  * `POST_CODE` becomes `addr:postcode` (or `addr:postc` temporarily) as an integer
+  * Manually add `addr:state` = `'FL'`
 * For multi-modal trails (golf cart paths):
   * `bicycle=yes`
   * `foot=yes`
@@ -29,6 +34,27 @@ See [https://wiki.openstreetmap.org/wiki/The_Villages_Road_and_Address_Import](h
   * `segregated=no`
   * `surface=asphalt`
 * Use the Filter with Form function to Select all entries with `"LIFECYCLE"='Current'`
+
+### For Lake County:
+
+* For roads:
+  * `FullStreet` becomes the virtual `name` via the `title(formatstreet("FullStreet"))`
+  * `SpeedLimit` becomes the virtual `maxspeed` via `concat("SpeedLimit",' mph')`
+  * `NumberOfLa` becomes the virtual `lanes`
+  * `surface=asphalt` added manually
+  * `StreetClas` becomes the virtual `highway` via the `gethighwaytype("StreetClas")`
+  * Could use MaxWeight (1.0 - 20.0)
+* For addresses:
+  * The Addresses shapefile is recorded in the NAD83(HARN) / Florida East (ftUS) CRS, you may need to convert or reproject to/from the default EPSG:4326 - WGS 84 CRS that OSM uses.
+  * `AddressNum` becomes the virtual `addr:housenumber` (or `addr:house` temporarily, avoiding addr:house which is a real tag) as an integer
+  * `UnitType` becomes the virtual `addr:unit` via `regexp_replace("UnitType",'U ','')` (UnitNumber is blank) as a string
+  * The virtual `addr:street` (or `addr:stree` temporarily) is created via the `regexp_replace(trim(concat(formatname("PrefixDire"),' ',title(formatstreet("PrefixType")),' ',title(formatstreet("BaseStreet")),' ',formatname("SuffixType"))),'\\s+',' ')` custom expression as a string
+  * `PostalCity` becomes the virual `addr:city` via the `title("PostalCity")` expression (we care about postal community addresses not what municipality a place might be governed by) as a string
+  * `ZipCode` becomes `addr:postcode` (or `addr:postc` temporarily) as an integer
+  * Manually add `addr:state` = `'FL'`
+
+### Continuing instructions for both:
+
 * Export to Geojson, only exporting **selected** entries, **selecting only the OSM-formatted fields we want**.
 * Here you can rename temporary columns like `addr:house` to `addr:housenumber`.
 * Ensure the export file is in the `EPSG:4326 - WGS84` CRS.

processed data/Lake/Addresspoints.qmd

@@ -0,0 +1,44 @@
+<!DOCTYPE qgis PUBLIC 'http://mrcc.com/qgis.dtd' 'SYSTEM'>
+<qgis version="3.30.1-'s-Hertogenbosch">
+  <identifier></identifier>
+  <parentidentifier></parentidentifier>
+  <language></language>
+  <type>dataset</type>
+  <title></title>
+  <abstract></abstract>
+  <contact>
+    <name></name>
+    <organization></organization>
+    <position></position>
+    <voice></voice>
+    <fax></fax>
+    <email></email>
+    <role></role>
+  </contact>
+  <links/>
+  <dates/>
+  <fees></fees>
+  <encoding></encoding>
+  <crs>
+    <spatialrefsys nativeFormat="Wkt">
+      <wkt>PROJCRS["NAD83(HARN) / Florida East (ftUS)",BASEGEOGCRS["NAD83(HARN)",DATUM["NAD83 (High Accuracy Reference Network)",ELLIPSOID["GRS 1980",6378137,298.257222101,LENGTHUNIT["metre",1]],ID["EPSG",6152]],PRIMEM["Greenwich",0,ANGLEUNIT["Degree",0.0174532925199433]]],CONVERSION["unnamed",METHOD["Transverse Mercator",ID["EPSG",9807]],PARAMETER["Latitude of natural origin",24.3333333333333,ANGLEUNIT["Degree",0.0174532925199433],ID["EPSG",8801]],PARAMETER["Longitude of natural origin",-81,ANGLEUNIT["Degree",0.0174532925199433],ID["EPSG",8802]],PARAMETER["Scale factor at natural origin",0.999941176470588,SCALEUNIT["unity",1],ID["EPSG",8805]],PARAMETER["False easting",656166.666666667,LENGTHUNIT["US survey foot",0.304800609601219],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["US survey foot",0.304800609601219],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["(E)",east,ORDER[1],LENGTHUNIT["US survey foot",0.304800609601219,ID["EPSG",9003]]],AXIS["(N)",north,ORDER[2],LENGTHUNIT["US survey foot",0.304800609601219,ID["EPSG",9003]]]]</wkt>
+      <proj4>+proj=tmerc +lat_0=24.3333333333333 +lon_0=-81 +k=0.999941176470588 +x_0=200000 +y_0=0 +ellps=GRS80 +units=us-ft +no_defs +type=crs</proj4>
+      <srsid>0</srsid>
+      <srid>0</srid>
+      <authid></authid>
+      <description>NAD83(HARN) / Florida East (ftUS)</description>
+      <projectionacronym></projectionacronym>
+      <ellipsoidacronym>PARAMETER:6378137:6356752.31414035614579916</ellipsoidacronym>
+      <geographicflag>false</geographicflag>
+    </spatialrefsys>
+  </crs>
+  <extent>
+    <spatial maxz="0" maxy="0" maxx="0" dimensions="2" crs="" minx="0" miny="0" minz="0"/>
+    <temporal>
+      <period>
+        <start></start>
+        <end></end>
+      </period>
+    </temporal>
+  </extent>
+</qgis>

qgis-functions.py

@@ -1,109 +1,259 @@
-from qgis.core import *
+import qgis.core
-from qgis.gui import *
+import qgis.gui
 import re
 
-@qgsfunction(args='auto', group='Custom', referenced_columns=[])
+#
-def getformattedstreetname(value1, feature, parent):
+# This will keep street names like SR 574A as SR 574A however
-    parts = value1.split()
+# will lowercase other number-digit suffixes with <2 or >4 numbers
-    parts = map(formatstreetname, parts)
+# or >1 suffix-letters, like 12th Street or 243rd Ave.
-    return " ".join(parts)
+#
 
 @qgsfunction(args='auto', group='Custom', referenced_columns=[])
-def getformattedstreetnamefromaddress(value1, feature, parent):
+def getstreetfromaddress(value1, feature, parent):
     parts = value1.split()
     parts.pop(0) # Ignore the first bit (i.e. "123" in "123 N MAIN ST")
     parts = map(formatstreetname, parts)
     return " ".join(parts)
 
+@qgsfunction(args='auto', group='Custom', referenced_columns=[])
+def formatstreet(value1, feature, parent):
+    parts = value1.split()
+    # Handle the special case of a street name starting with "ST"
+    # which is almost always "Saint __" and not "Street __"
+    if parts[0].upper() == "ST":
+        parts[0] = "Saint"
+    if parts[0].upper() == "ROYAL" and parts[1].upper() == "ST":
+        parts[0] = "Royal"
+        parts[1] = "Saint"
+    # And "CR" as a first part (County Road) vs last part (Circle)
+    if parts[0].upper() == "CR":
+        parts[0] = "County Road"
+    if parts[0].upper() == "SR":
+        parts[0] = "State Route"
+    parts = map(formatstreetname, parts)
+    return " ".join(parts)
+
+
+@qgsfunction(args='auto', group='Custom', referenced_columns=[])
+def formatname(value1, feature, parent):
+    parts = value1.split()
+    parts = map(formatstreetname, parts)
+    return " ".join(parts)
+
+@qgsfunction(args='auto', group='Custom', referenced_columns=[])
+def gethighwaytype(value1, feature, parent):
+    match value1:
+        case "ALLEY":
+            return "alley"
+        case "LOCAL":
+            return "residential"
+        case "MAJOR":
+            return "trunk"
+        case "MEDIAN CUT":
+            return "primary_link"
+        case "OTHER":
+            return "unclassified"
+        case "PRIMARY":
+            return "primary"
+        case "PRIVATE":
+            return "service"
+        case "RAMP":
+            return "trunk_link"
+        case "SECONDARY":
+            return "secondary"
+        case "TURN LANE":
+            return "primary_link"
+        case "VEHICULAR TRAIL":
+            return "track"
+
+# Internal function
 def formatstreetname(name):
-    # Specific suffixes like "123th" we have lower
+    nameUp = name.upper()
-    if re.search("[0-9]+TH", name):
+    # Acronyms
-        return name.capitalize()
+    if nameUp == "SR":
-    if re.search("[0-9]+ND", name):
+        return "SR" # State Route
-        return name.capitalize()
+    if nameUp == "NFS":
-    if re.search("[0-9]+ST", name):
+        return "NFS" # National Forest Service?
-        return name.capitalize()
+    if nameUp == "US":
-    if re.search("[0-9]+RD", name):
-        return name.capitalize()
-    # Weird names like 123D we keep upper
-    if re.search("[0-9]+[A-Z]+", name):
-        return name
-    # Highway/etc prefixes
-    if name == "US":
         return "US"
-    if name == "SR":
-        return "SR"
-    if name == "CR":
-        return "County Road"
-    if name == "C":
-        return "C"
     # Directions
-    if name == "N":
+    if nameUp == "N":
         return "North"
-    if name == "NE":
+    if nameUp == "NE":
         return "Northeast"
-    if name == "E":
+    if nameUp == "E":
         return "East"
-    if name == "SE":
+    if nameUp == "SE":
         return "Southeast"
-    if name == "S":
+    if nameUp == "S":
         return "South"
-    if name == "SW":
+    if nameUp == "SW":
         return "Southwest"
-    if name == "W":
+    if nameUp == "W":
         return "West"
-    if name == "NW":
+    if nameUp == "NW":
         return "Northwest"
+    # Names
+    if nameUp == "MACLEAY":
+        return "MacLeay"
+    if nameUp == "MCCLAINE":
+        return "McClaine"
+    if nameUp == "MCAHREN":
+        return "McAhren"
+    if nameUp == "MCCAMMON":
+        return "McCammon"
+    if nameUp == "MCCLELLAN":
+        return "McClellan"
+    if nameUp == "MCCOY":
+        return "McCoy"
+    if nameUp == "MCDONALD":
+        return "McDonald"
+    if nameUp == "MCGEE":
+        return "McGee"
+    if nameUp == "MCGILCHRIST":
+        return "McGilchrist"
+    if nameUp == "MCINTOSH":
+        return "McIntosh"
+    if nameUp == "MCKAY":
+        return "McKay"
+    if nameUp == "MCKEE":
+        return "McKee"
+    if nameUp == "MCKENZIE":
+        return "McKenzie"
+    if nameUp == "MCKILLOP":
+        return "McKillop"
+    if nameUp == "MCKINLEY":
+        return "McKinley"
+    if nameUp == "MCKNIGHT":
+        return "McKnight"
+    if nameUp == "MCLAUGHLIN":
+        return "McLaughlin"
+    if nameUp == "MCLEOD":
+        return "McLeod"
+    if nameUp == "MCMASTER":
+        return "McMaster"
+    if nameUp == "MCNARY":
+        return "McNary"
+    if nameUp == "MCNAUGHT":
+        return "McNaught"
+    if nameUp == "O'BRIEN":
+        return "O'Brien"
+    if nameUp == "O'CONNOR":
+        return "O'Connor"
+    if nameUp == "O'NEIL":
+        return "O'Neil"
+    if nameUp == "O'TOOLE":
+        return "O'Toole"
     # Suffixes
-    if name == "AVE":
+    if nameUp == "ALY":
+        return "Alley"
+    if nameUp == "AV":
         return "Avenue"
-    if name == "BLVD":
+    if nameUp == "AVE":
+        return "Avenue"
+    if nameUp == "BAY":
+        return "Bay"
+    if nameUp == "BLF":
+        return "Bluff"
+    if nameUp == "BLVD":
         return "Boulevard"
-    if name == "BND":
+    if nameUp == "BV":
+        return "Boulevard"
+    if nameUp == "BND":
         return "Bend"
-    if name == "CIR":
+    if nameUp == "CIR":
         return "Circle"
-    if name == "CT":
+    if nameUp == "CR":
+        return "Circle"
+    if nameUp == "CRK":
+        return "Creek"
+    if nameUp == "CRST":
+        return "Crest"
+    if nameUp == "CT":
         return "Court"
-    if name == "DR":
+    if nameUp == "CURV":
+        return "Curve"
+    if nameUp == "CV":
+        return "Curve"
+    if nameUp == "DR":
         return "Drive"
-    if name == "FLDS":
+    if nameUp == "FLDS":
         return "Fields"
-    if name == "GRV":
+    if nameUp == "GLN":
+        return "Glenn"
+    if nameUp == "GRV":
         return "Grove"
-    if name == "HOLW":
+    if nameUp == "HL":
+        return "Hill"
+    if nameUp == "HOLW":
         return "Hollow"
-    if name == "HWY":
+    if nameUp == "HTS":
+        return "Heights"
+    if nameUp == "HW":
         return "Highway"
-    if name == "LN":
+    if nameUp == "HWY":
+        return "Highway"
+    if nameUp == "HY":
+        return "Highway"
+    if nameUp == "LN":
         return "Lane"
-    if name == "LOOP":
+    if nameUp == "LNDG":
+        return "Landing"
+    if nameUp == "LOOP":
         return "Loop"
-    if name == "PATH":
+    if nameUp == "LP":
+        return "Loop"
+    if nameUp == "MNR":
+        return "Manor"
+    if nameUp == "MT":
+        return "Mount"
+    if nameUp == "MTN":
+        return "Mountain"
+    if nameUp == "PARK":
+        return "Park"
+    if nameUp == "PASS":
+        return "Pass"
+    if nameUp == "PATH":
         return "Path"
-    if name == "PL":
+    if nameUp == "PKWY":
+        return "Parkway"
+    if nameUp == "PL":
         return "Place"
-    if name == "RD":
+    if nameUp == "PLZ":
+        return "Plaza"
+    if nameUp == "PS":
+        return "Pass"
+    if nameUp == "PT":
+        return "Point"
+    if nameUp == "RD":
         return "Road"
-    if name == "RDG":
+    if nameUp == "RDG":
         return "Ridge"
-    if name == "RUN":
+    if nameUp == "RUN":
         return "Run"
-    if name == "ST":
+    if nameUp == "SHRS":
+        return "Shores"
+    if nameUp == "SQ":
+        return "Square"
+    if nameUp == "ST":
         return "Street"
-    if name == "TER":
+    if nameUp == "TER":
         return "Terrace"
-    if name == "TRL":
+    if nameUp == "TR":
         return "Trail"
-    if name == "VW":
+    if nameUp == "TRL":
+        return "Trail"
+    if nameUp == "VW":
         return "View"
-    if name == "WAY":
+    if nameUp == "WALK":
+        return "Walk"
+    if nameUp == "WAY":
         return "Way"
-    if name == "XING":
+    if nameUp == "WY":
+        return "Way"
+    if nameUp == "XING":
         return "Crossing"
-    # Irish names
+    if re.match('^[0-9]{2,4}[A-Za-z]$', name) != None:
-    if name == "MCCRAY":
+        return name
-        return "McCray"
+
-    if name == "MCKOWN":
+    return name #.capitalize()
-        return "McKown"
-    return name.capitalize()