Friday, October 9, 2020

TouchTerrain 3.0: New GUI, place search, kml polygon file upload

What's new in Version 3? 

New GUI: With version 3.0 (October 2020), TouchTerrain has gotten a serious facelift. Gone is the utilitarian ca. 1995 aesthetic, replaced by a Bootcamp 4 based GUI. This is my first ever dip into using bootcamp and there are probably still some details to be worked out. 

GUI responsiveness: The Bootcamp GUI implements what it calls responsiveness which means that (to an extent) the app will try to dynamically"compact" itself when used with small screens (tablets, smartphones). Note that this should work on a smartphone but will look a bit weird. Hopefully most users will have a reasonably wide display when using the app!

GUI size adjustments: You will notice that there are 3 main parts ("cards" in Bootstrap speak) to the right of the map. Clicking on their titles (e.g. on Area Selection Box:) will grow or shrink them. By default, only the Terrain Settings and the 3D Printer Options parts are "open", the Area Selection Box (which hides the lat/long coordinates of the box corners) is closed. Opening all three together will make the green Export Button on the bottom to be shoved out of sight, so you'll have to close a part or scroll down to click on it.

Manual box coordinate entry: In version 3, the four coordinate fields inside Area Selection Box are now active. Changing any of the numbers and hitting Enter will immediately resize the red box accordingly.

"Oversampling" warning: If you configure things in a way that TouchTerrain will interpolate the DEM into a higher (more detailed) resolution that what the DEM source can provide, the CUrrent DEM resolution value will turn yellow. Ex: The source DEM resolution is 30 m but you're requesting 20 m. This will still "work" but it's counterproductive b/c it won't magically create more detail and your files will be needlessly larger than if you had configured it to resample at say 32 m instead.

Place search bar: As is standard in many other Google Map applications, you can now search for a place by typing a search term into the box and pick one of the five suggestions displayed.  Your map view will fly to the result you picked. Be aware that it will NOT move the red selection box to your new view, so don't forget to hit the blue Re-center box button when needed!

Use a Polygon from a kml file: If you have a kml file that contains a polygon or a polyline (e.g. the outline of a state) you can upload it to the app and use it as boundaries for the 3D model you download. 

To upload the kml file, click on Browse, select a kml file from you local drive and hit Open. Note that is has to be kml file, not a kmz file! Kmz is just the zipped version of kml. To use a kmz, unzip the kmz file, which will give you a file called doc.kml, just rename that and then upload it. You could of course also use Google Earth to digitize a polygon, save it into a kml file and upload it to the app.

After the upload your map view will jump to the area and will see your polygon in yellow inside the red box. Adjust the other settings and then hit Export. You will see a note (Using X points from kml file as polygon) and you model (and geotiff file) will be clipped (masked) by the polygon.

Note that this polygon will be cleared if you move the red box or if you use functionality that requires the app to reload, such as changing the DEM source or the hillshade settings. This is needed, b/c the polygon isn't inlined into the URL (like all the other settings). But, it should be pretty quick to re-upload the kml file again! Finally, be aware that if your file contains multiple polygons or polylines, only the first will be taken.

Other changes:

  • URL: Depending on you system you may no longer see the full URL in the browser. However, we now give you the full URL at the end of the export (as simple browser text), so you can still use it to "give" your model to somebody else.
  • GPX path lines:  On the standalone version of TouchTerrain, you can now drape path lines from gpx files over your model.
  • Map with jupyter notebook: The jupyter notebook standalone version can now show you an interactive Google Map in a notebook cell (using the geemap module). It also lets you digitize a rectangle, circle or polygon onto that map and use it as your model area.
























Wednesday, May 13, 2020

touchterrain 2.5: New Hillshading, more DEM sources, lower_leq setting and new folder structure (May 13)

The May 13 release of touchterrain (v. 2.5) contains a whole bunch of new and changed features:

New Hillshading

  • Hillshading creates a overlay map that shows a pseudo-3D relief of the underlying DEM. This is useful as a pre-visualization of how your 3D print will roughly look and helps users to select interesting areas to print.
  • Hillshading (in its simplest form) assumes that a sun is positioned above the terrain at certain angles:
    • Azimuth: Defines the direction (compass heading) from which the Sun shines to create the hillshade relief. Note that South will typically result in a flipped relief perception!
    • Elevation: Defines the vertical Sun angle, i.e. how high the Sun is above the horizon. Use it to enhance detail for different types of terrain.
  • So basically we have a 2 angles, a horizontal (0 - 360, 0 = North) and vertical (0 - 90, 90 = perfectly above) that define the position of the light source used to create the impression of a 3D relief by creating greyscale values from 0 (dark) to 255 (super bright). Note that this does not create shadows, only the effect of specular highlights, which is a very strong perceptional cues for creating a 3D effect.
  • The standard hillshade parameter values are: azimuth of 315° (NNE) and 45° for elevation, which are good all-around settings.
  • However, it can be fun (and rewarding) to change these angles, which is what the new version allows you to do (in a limited fashion)!
  • For example, for best results the Sun should come at an angle roughly perpendicular to ridges in your terrain. Here's a comparison for different azimuth values (elevation angle is 60) (source):

  • 270, 315, 0 and 45 all bring out different aspects of the same data, e.g. 45 emphasizes NE-SW oriented features. Also, to me, some values seem to make my brain reverse the topography i.e. ridges look like valleys and vice versa. 
  • Looking at the elevation value, values lower than 45, create darker and darker overlays but also more and more emphasize low relief terrain features, such as drainage channels (but at the cost of blowing out high relief features).
  • For touchterrain 2.5 I have attempted to implement a simple way to experiment with these two hillshade angles without overwhelming and confusing the user:
  • Sun direction (new) has a few direction presets shown as words (West, North-West, etc.). I only sweep the northern directions in 45 steps b/c I'm pretty sure that using southern direction would result in relief confusion. But, just to force the effect I include South as well.
  • I renamed the Elevation parameter to Sun Angle (b/c I want to avoid confusion with terrain elevation) and again offer just a few presets: Normal (45), Steep (55) and several progressively flatter settings to go below 45 all the way to 5
  • Finally there's a gamma value via a direct number input. This gamma correction factor is used to somewhat compensate that low sun angles tend to create hillshades that look too dark.  Gamma values > 1 increase the "lightness" so as the sun angle becomes lower, this gamma will increase to compensate. Note that the specific gamma values are my "opinion" only! If you want to tweak the gamma after selecting a sun angle preset, just change the number and hit enter. Feel free to experiment, but be aware that your tweak will be overwritten next time you choose another sun angle preset!
  • Be aware that changing any of the hillshade parameters will require a reload of the map, b/c I need to give Earth Engine these new values, get back a new hillshade overlay calculated for the new values and then display it. So it might take a split second, depending on your internet.
  • Note that I have tried to preserve all other input values you might have chosen before doing such a reload. This includes the type of Google Map map in the background and the transparency. However, this is not fully tested, so please let me know if values for some fields you changed earlier are getting reset.
  • If you're curious, go and change the Google Map to Terrain (Map -> Terrain), which also has hillshading backed in, and lower your transparency to mix them both.
  • Transparency (inverse of opacity) is still given in percent but I made the slider much, much smoother (1% steps). I also removed the direct numeric input b/c I feel having a precise number value is not how transparency is used, user should simple move the slider until they are happy.  Changing Transparency will not trigger a reload, b/c it's set via your browser (client) not via the server.
  • Finally, there are a bunch of new query parameters in the URL that store the new hillshade stettings and transparency: transp= 0 -100,  hsazi= -360 - 360 (hs azimuth), hselev=  0 - 90 (hs elevation) and gamma= 0 - 99, so if you need values that cannot be generated in the GUI, just change those in the URL and hit enter. Note that none of these parameters matter when processing the terrain into 3D models, they are purely for a better visual experience.
  • So ... got nuts and play with these values!  Set the transparency to 0 (full left) and set the Sun angle to extremely flat. I think this looks very interesting (or at least artistic :)

More DEM sources

  • I have added a few new sources and added a link to their meta data (via earth engine):
    • USGS/NED: 10 m, continental USA only. link
    • ALOS DSM: Global: 30 m, worldwide, but has some small "holes". link
    • USGS/SRTMGL1_003: 30 m, "worldwide", but not very far north (lower quality and older than ALOS!). link
    • USGS/GMTED2010: ~230 m, truly worldwide. link
    • GTOPO30: 30 Arc-Second, 1000 m, 1996, worldwide. link
    • CryoSat-2 Antarctica: 1000 m, antarctica only. link
    • NOAA/NGDC/ETOPO1: 2000 m, worldwide, with bathymetry. link
  • For any non-US area ALOS should be your preferred choice as it's much newer and of better quality. However, it misses data in a few tiny areas ("holes")
  • Using the Antarctica data is sadly a bit silly at the moment b/c Google maps are way too distorted.

lower_leq setting
































  • Thanks to Iden Craven a manual setting was added to lower any elevation values that are lower than a threshold (say <= 0 m, i.e. anything not land) by a millimeter value when printed out.
  • Example "lower_leq":[0, 5] as a manual setting will lower offshore areas by 5 mm.
  • This can emphasize shorelines while still printing area below sealevel. (ignore_leq would completely omit these areas when printed)







 New folder structure

  • I decided to create a proper module for most of the touchterrain code, which required a new folder structure:
    • standalone was remove and the two "main" files TouchTerrain_standalone.py, and its jupyter notebook version, TouchTerrain_standalone_jupyter_notebook.ipynb (recommended) are now at the root level.
    • All other code (the common and server folders) was moved into a new folder called touchterrain
    • Before running the stand alone version, a module called touchterrain needs to be build and installed in your system's site-packages folder. This is done via setup.py, best by using pip: pip install . <-dot!
    • The jupyter notebook now explains the installation and can even run it inside a cell as a shell command!
    • auxiliary files (examples, etc.) were mover to a new folder called stuff

Tuesday, March 31, 2020

Adapted to Earth Engine API changes (Mar. 31)


  • Google announced a while back that they are changing the JS and Python API to be supported by it's Cloud API
  • For me, that change broke several things, both in Python and in JS:
    • I was suddenly unable to d/l any geotiffs from EE to make models from, which took time to figure out.
    • Even then, none of my hillshade overlays showed up, again, that took some time to fix.
  • Impact: 
    • There was an outage of a couple of days that prevented any d/l. Several users got in contact with me; if you've been affected - I'm sorry!
    • But, nobody complained about the lack of hillshade overlays :)
    • Google has apparently decided to lower the maximum size that can be downloaded. It should not affect TouchTerrain, b/c I only request appropriately downsampled geotiffs from them anyway and our server also caps how many pixels you can process on it, but I have not tested this well. It's possible that users will run into the Google quota if they request very large models.
    • But, that pretty much kills the "download at source resolution" option I had and so I took it out. It's still available in standalone to make models at the exact same resolution as a local raster (geotiff). 
    • It is still possible to "export" large rasters into something running in the Google Cloud but I have no intentions to run our server in the Google cloud. At least not for the foreseeable future.
  • Silver lining: After going back to digging around the Earth Engine API I noticed a whole bunch of new terrain and terrain related datasets!  My next plan is to add some of these to Touchterrain.

Exporting Terrain models with real world coordinates


Exporting Terrain models with real world coordinates

  • Some people would like to import TouchTerrain model files (STLs or OBJs) into software for CAD or 3D Modelling. 
  • At my University (Iowa State), an architecture student imported terrain into Rhino, manually mach a building he designed to its correct place and CNC'd the entire thing!
  • Another request asked for the ability to import a TouchTerrain STL into Blender in order to make changes to it there. Specifically, to combine it with a GPS track in order to make the track visible when the 3D terrain is printed out (more about this later, provided this turns out to have worked ...): Update: this is now an official touchterrain option (thanks to KohlhardtC!), see the gpx option in the github ReadMe.

  • As a result, I decided to add an option to set the xyz coordinates of the vertices inside the STL/OBJ files as something in a global context. (Typically, the coordinates are in mm and the origin is the center of your buildplate ...)
  • Using unprojected lat/long coordinates is out of the question for many reasons (distortions, height, sphere vs plane, etc.), so the only real option is using UTM coordinates, which TouchTerrain already requests from EarthEngine. This makes the x/y coordinates be large numbers, something in 100,000s or even millions, with z (elevation) always in meters. 
  • These UTM x/y coordinates are distances from the intersection of the equator with Greenwich. Here's a web app that show those numbers for the (nearly) entire globe.
  • In theory, terrain models with these numbers, when imported into a 3D modelling system, should show up in the correct place and should also be correct with regard to other, UTM encoded, objects.
  • Sadly, the only system I remotely know how to use is Blender, for which these standard UTM coordinates did NOT work (at least for me)!

  • However, after looking into BlenderGIS, I realized that I could trick Blender in positioning the STL model in the correct place, if I first imported the geotiff of the terrain (which is always given in the zip folder you download). The trick seems to be to still use meters as units but set the origin (0,0,0) at the center of the geotiff.  For this to work, you need to use the "centered" option (see below).
  • I've documented my Blender adventure here (very detailed, illustrated instruction pdf and a zip with example files used). It only deals with draping gpx lines on the topography (which was added as an official option to standalone in version 3) but it will also show you how to center a touchterrain model for use with BlenderGIS. So I imagine this method should also work for placing other types of geometry (like buildings) onto the model using Blender.

  • Using ArcGIS Pro, I was able to import a OBJ terrain mesh from TouchTerrain (with the "UTM" option, see below) into a local 3D Scene and it does overlap nicely with its geotiff. I used the Import 3D Files (3D Analyst) function.
  • To show the fit, I've set the build in terrain to flat. The "shadow" over which the imported OBJ seems to be floating is the geotiff.
TouchTerrain 3D model of Sheep Mountain, imported as OBJ into ArcGIS Pro, local 3D Scene

  • So what now? Well, if you want to try using a TouchTerrain Model in the context of a 3D Modelling software, all you need to do is to set the manual option:
    • "use_geo_coords":"centered" will set the UTM origin to the center of the full tile
    • "use_geo_coords":"UTM" will use the official UTM x/y coordinates
  • Again, this is highly experimental and only tested in Blender (with "centered" and using BlenderGIS to import the geotiff first!) and ArcGIS Pro. If it works for you in another 3D software, please let me know! (If it fails, also let me know, maybe I can make it work ...)

Monday, September 16, 2019

TouchTerrain 2.0 is out

TouchTerrain version 2.0 is now live on https://touchterrain.geol.iastate.edu. Big thank you to the beta-testers! https://touchterrain-beta.geol.iastate.edu will still remain active but will be my personal development server where I implement new stuff, which may break the server so you should not use it anymore. Speaking of breaking, if you are experiencing issues (page hangs, not download, etc.) please let us know via Geofablab@gmail.com, same with questions and suggestions! If you are reporting a bug please tell us the 4 digit version shown in the title, e.g. (0914).



Summary of changes and addition for version 2.0

(also here on github)

3D Preview: Once your model has been processed, you can preview it in 3D (rotate, zoom, etc.) right in your browser. This uses www.viewstl.com's Javascript plugin which uses Three,js (WebGL) for rendering. As everything happens inside your browser, it may take a while to actually show you the graphics, especially for large models.

Snazzy wait-animation: I still can't show you how far your processing has gone, but at least now you get a nice animation while you wait. Also, there's no preflight screen anymore, as soon as you hit Export, processing will commence. It's still recommended to hit Save to URL before you hit Export, otherwise your current settings won't get save to the URL and you might need them later!

Optional feedback/survey text box: I've added a textbox just above the download button where users can tell us what they use the terrain models for (150 chars or less). This is completely optional (you can just leave the box empty) and no personal info is stored. However, we would appreciate it if you write a quick note, so that we, over time, can get a rough idea what the use cases for 3D printed terrain models are.

Geotiffs: you will now always get a geotiff of the area that you requested (red box) but projected into UTM at the resolution used to created the STL(s). E.g. if you requested a print resolution of 0.4 mm and this translated into say 27.6 m for each cell, this will be the resolution of the geotiff. If you requested several tiles, the geotiff will be the pre-tiled version i.e. you'll only get one geotiff, not many.

Geotiff as File Format: This will only give you the Geotiff as output. Use this with print resolution set to source resolution to grab the geotiff at its native resolution. This should be very handy for users who just want a geotiff (e.g. for GIS work) but need the best possible resolution.

Print resolution (printres) set to source resolution: every DEM source has a "native" resolution (e.g. ~10 m for NED). Using source resolution for the print resolution instead of a mm number, will use this native resolution and give you the highest possible level of detail for your 3D terrain model. 
This is primarily  meant for the geotiff-only output, where it ensures that no interpolation due to re-sampleing is done. In other words, you'll pretty much get the original data except it will be projected to UTM. E.g. for NED it'll be the same data as you'd get from the USGS TNM website and you don't have to d/l a full 1 degree block and clip out your area.
For STL/OBJ, using the source resolution will result in the highest detail possible for the 3D models albeit at the cost of very large files, so you'll have to be content with relatively small areas or run up against our server limits. (Note: if you run your own server or use the stand-alone version, you can go nuts with this, just don't be surprised if processing takes a long time and gives you huge files :)

More precise model dimensions: I spend some time making sure that the area you select and the model you actually get are the same. The tricky part is that the red selection box provides four corners, each given in lat/long. However, the lat/long box is always more or less a trapezoid, not a rectangle (thanks, "round" Earth!) but the geotiff I get from EarthEngine is in UTM (meter based) and is always a rectangle.
I did work to improve this by adjusting the print resolution a tiny bit if needed. Previously, a printres of 0.4 might have resulted in a raster is 234.576 cells wide, but as there are no partial cells it had to be shortened to 234 cell and thus the model was a bit too small. Now, I adjust the printres (to say 0.3994554) so that the resulting raster exactly 235 cells wide. This also improves the fit of tiles. Previously it was possible that the tiles were a bit too short and when you assembled them you could see small offset artifacts were they touched. 
The log file will show when such adjustments were made and what your defacto print resolution and model dimensions are, e.g.:

cell size: 109.05937041208486 m  
adjusted print res from the requested 0.5 mm to 0.4784688995215311 mm to ensure correct model dimensions 
total model size in mm: 100.0 x 99.52153110047847 
Cropping for nice fit of 2 (width) x 2 (height) tiles, removing: 1 columns, 0 rows 
 cropped (209, 208) to (208, 208) 
 cropping changed physical size from 50.0 mm x 49.760765550239235 mm 
 to 49.760765550239235 mm x 49.760765550239235 mm 
map scale is 1 : 227934.08416125734 

Server Limits: We're still tuning those but the current limits should allow models of up to 300 x 300 mm at 0.4 mm resolution (or 3 x 3 tiles of 100 x 100 mm each, etc.). This should work for areas at the equator. As the farther you get towards a pole, the less data you're actually requesting, you should actually be able to get even larger models away from the equator. If you're over the limit, you will be told by how much, so you can go back and adjust your area. We think that's a pretty generous model size, given that most common  3D printers don't even have 300 x 300 mm buildplates (CR-10s, etc. non-withstanding).  If you need multiple tiles of that size, there's a "cheat" you can use (see below).

Which brings us to the manual options:

Manual options: There's now a large field beneath file format to manually put in options and values. Some options are not part of the UI options , but you can override UI options! To put in an option and an override value, type in the name of the option in double quotes, then a colon, then the value (no spaces). If you want to add other options, add a comma and space and repeat. The name of an UI option is shown when you mouse over the UI part: it's the first word in parens.

Example:
  • You want a print resolution of 0.37 mm, not 0.4 or 0.35 
  • The name of that options is printres, so you type in:
  • "printres":0.37
  • You also want a model width of 175 mm, name for this is tilewidth, so you add that:
  • "printres":0.37, "tilewidth":175

 Now for the fun part, where we use the new, non-UI options:

  • tile_centered: default: false. With false, all tiles are offset so they all "fit together" when they all are loaded into a 3D viewer, such as Meshmixer or Meshlab. Note that slicers (e.g. Cura) have ways to put the model into the center of the build plate, even if its center coordinates is not 0,0. Set this to true to force all tile STL to be centered around 0/0: "tile_centered":true
  • no_bottom: default: false. Will omit any bottom triangles i.e. only stores the top surface and the "walls". This creates ~50% smaller STL/OBJ files. When sliced it should still create a solid printed bottom (tested in Cura). This option is false by default. meaning you will get all the bottom triangles unless you overwrite it with true:  "no_bottom":true
  • no_normals: default: true. Will NOT calculate normals for triangles in STL files and instead set them to 0,0,0. This is significantly faster and should not matter as on import most slicers and 3D viewers will calculate the normal for each triangle (via cross product) anyway. But, if you require properly calculated normals in your STL, set it to false: "no_normals":false
  • ignore_leq: default: null. Using an elevation (e.g. 0.0), this  will ignore any cells less or equal to that elevation. Good for omitting offshore cells and print only onshore terrain. Note that 0 may not be exactly sealevel on some DEMs, you may have to experiment with slightly higher values to remove all of the water, e.g. 0.5: "ignore_leq":0.5
  • projectiondefault: null. By default, the DEM is reprojected to the UTM zone (datum: WGS84) the model center falls into. The EPSG code of that UTM projection is shown in the log file, e.g. UTM 13 N, EPSG:32613. If a number(!) is given for this projection setting, the system will request the Earth Engine DEM to be reprojected into it. For example, maybe your data spans 2 UTM zones (13 and 14) and you want UTM 14 to be used, so you set projection to 32614. Or maybe you need to use UTM 13 with NAD83 instead of WGS84, so you use 26913. For continent-size models, WGS84 Web Mercator (EPSG 3857), my work better than UTM. See https://spatialreference.org for descriptions of EPSG codes. Be aware, however, that Earth Engine does not support all possible EPSG codes. For example, North America Lambert Conformal Conic (EPSG 102009) is not supported and gives an error message: The CRS of a map projection could not be parsed
  • only: Giving two tile index numbers [x,y], this will only process one tile and ignore all other tiles. [1,1] is the upper left tile, [2,1] it the tile to it's right, etc. This will enable users to d/l otherwise unreasonably large models by processing one of its tiles at a time (thus staying under the server limit), then manually selecting the next tile and process it, etc.

    Example:  You have 4 tiles, each 250 mm wide, so you can't process all 4 of them at once b/c of the server limits. Add the option "only":[1,1] and make sure to Save to URL before hitting Export. This will only process the tile with index 1,1. Once this tile was downloaded, go back to the main screen and change it to "only":[2,1], repeat for all 4 tiles. 
    Note that each tile will 
    be in a new zip, so you should unzip all of them in a common folder. In a 3D viewer, the tiles will fit together without overlaps if tile_centered was false.

Standalone mode

There are now 2 ways to run TouchTerrain on your own local PC/Mac/Linux box. After you get teh code from github look in the standalone folder There is is a python file called TouchTerrain_standalone.py that you can run via CLI or IDE. It does require a JSON config file as argument, which contains the equivalent of the settings from the web app GUI. However, it might be easier to run the jupyter notebook TouchTerrain_standalone_jupyter_notebook.ipynb and hand edit the config setting in it and then run it (here is a html preview of it). This will also give you a in-browser 3D preview of your model(s).

Wednesday, June 12, 2019

TouchTerrain 2.0-beta is out!

TouchTerrain 2.0-beta is out!

Since Fall last year, I've been working on improvements and fixes  In some cases, I've already added them to the current version (1.2x) but most of them are new ... and untested by real users.  I therefore decided to have a beta phase with a separate beta server, as to not interrupt the old server. 

To try the beta version got to:

https://touchterrain-beta.geol.iastate.edu


Any feedback is very much appreciated, please use geofablab@gmail.com to email us any bugs, observations, requests, etc. Please note the version of the beta (see title, will be something like beta-0614) and add screenshots if needed. Thanks!

Finally, huge shoutout to Levi and Nick from ISU Research IT for setting up a docker-webhook-magic-thingy that should (theoretically) enable me to make fixes/changes to my code and update the server directly myself! This should results in much faster turnaround on fixes to the beta.


What's new in 2.0?

Despite not having gotten much of a face-lift, TouchTerrain 2.0 has a lot of new features behind the scenes! I also re-visited some areas that needed more attention and improved or at least cleaned up some issues.

3D Preview: Once your model has been processed, you can preview it in 3D (rotate, zoom, etc.) right in your browser. This uses www.viewstl.com's Javascript plugin which uses Three,js (WebGL) for rendering. As everything happens inside your browser, it may take a while to actually show you the graphics, especially for large models.

Snazzy wait-animation: I still can't show you how far your processing has gone, but at least now you get a nice animation while you wait. Also, there's no preflight screen anymore, as soon as you hit Export, processing will commence. It's still recommended to hit Save to URL before you hit Export, otherwise your current settings won't get save to the URL and you might need them later!

Geotiffs: you will now always get a geotiff of the area that you requested (red box) but projected into UTM at the resolution used to created the STL(s). E.g. if you requested a print resolution of 0.4 mm and this translated into say 27.6 m for each cell, this will be the resolution of the geotiff. If you requested several tiles, the geotiff will be the pre-tiled version i.e. you'll only get one geotiff, not many.

Geotiff as File Format: This will only give you the Geotiff as output. Use this with print resolution set to source resolution to grab the geotiff at its native resolution. This should be very handy for users who just want a geotiff (e.g. for GIS work) but need the best possible resolution.

Print resolution (printres) set to source resolution: every DEM source has a "native" resolution (e.g. ~10 m for NED). Using source resolution for the print resolution instead of a mm number, will use this native resolution and give you the highest possible level of detail for your 3D terrain model. 
This is primarily  meant for the geotiff-only output, where it ensures that no interpolation due to re-sampleing is done. In other words, you'll pretty much get the original data except it will be projected to UTM. E.g. for NED it'll be the same data as you'd get from the USGS TNM website and you don't have to d/l a full 1 degree block and clip out your area.
For STL/OBJ, using the source resolution will result in the highest detail possible 3D models albeit at the cost of very large files, so you'll have to be content with relatively small areas or run up against our server limits. (Note: if you run your own server or use the stand-alone version, you can go nuts with this, just don't be surprised if processing takes a long time and gives you huge files :)

More precise model dimensions: I spend some time making sure that the area you select and the model you actually get are the same. The tricky part is that the red selection box provides four corners, each given in lat/long. However, the lat/long box is always more or less a trapezoid, not a rectangle (thanks, "round" Earth!) but the geotiff I get from EarthEngine is in UTM (meter based) and is always a rectangle.
I did work to improve this by adjusting the print resolution a tiny bit if needed. Previously, a printres of 0.4 might have resulted in a raster is 234.576 cells wide, but as there are no partial cells it had to be shortened to 234 cell and thus the model was a bit too small. Now, I adjust the printres (to say 0.3994554) so that the resulting raster exactly 235 cells wide. This also improves the fit of tiles. Previously it was possible that the tiles were a bit too short and when you assembled them you could see small offset artifacts were they touched. 
The log file will show when such adjustments were made and what your defacto print resolution and model dimensions are, e.g.:

cell size: 109.05937041208486 m  
adjusted print res from the requested 0.5 mm to 0.4784688995215311 mm to ensure correct model dimensions 
total model size in mm: 100.0 x 99.52153110047847 
Cropping for nice fit of 2 (width) x 2 (height) tiles, removing: 1 columns, 0 rows 
 cropped (209, 208) to (208, 208) 
 cropping changed physical size from 50.0 mm x 49.760765550239235 mm 
 to 49.760765550239235 mm x 49.760765550239235 mm 
map scale is 1 : 227934.08416125734 

Server Limits: We're still tuning those but the current limits should allow models of up to 300 x 300 mm at 0.4 mm resolution (or 3 x 3 tiles of 100 x 100 mm each, etc.). This should work for areas at the equator. As the farther you get towards a pole, the less data you're actually requesting, you should actually be able to get even larger models away from the equator. If you're over the limit, you will be told by how much, so you can go back and adjust your area. We think that's a pretty generous model size, given that most common  3D printers don't even have 300 x 300 mm buildplates (CR-10s, etc. non-withstanding).  If you need multiple tiles of that size, there's a "cheat" you can use (see below).

Which brings us to the manual options:

Manual options: There's now a large field beneath file format to manually put in options and values. Some options are not part of the UI options , but you can override UI options! To put in an option and an override value, type in the name of the option in double quotes, then a colon, then the value (no spaces). If you want to add other options, add a comma and space and repeat. The name of an UI option is shown when you mouse over the UI part: it's the first word in parens.

Example:
  • You want a print resolution of 0.37 mm, not 0.4 or 0.35 
  • The name of that options is printres, so you type in:
  • "printres":0.37
  • You also want a model width of 175 mm, name for this is tilewidth, so you add that:
  • "printres":0.37, "tilewidth":175

 Now for the fun part, where we use the new, non-UI options:

  • tile_centered: default: false. With false, all tiles are offset so they all "fit together" when they all are loaded into a 3D viewer, such as Meshmixer or Meshlab. Note that slicers (e.g. Cura) have ways to put the model into the center of the build plate, even if its center coordinates is not 0,0. Set this to true to force all tile STL to be centered around 0/0: "tile_centered":true
  • no_bottom: default: false. Will omit any bottom triangles i.e. only stores the top surface and the "walls". This creates ~50% smaller STL/OBJ files. When sliced it should still create a solid printed bottom (tested in Cura). This option is false by default. meaning you will get all the bottom triangles unless you overwrite it with true:  "no_bottom":true
  • no_normals: default: true. Will NOT calculate normals for triangles in STL files and instead set them to 0,0,0. This is significantly faster and should not matter as on import most slicers and 3D viewers will calculate the normal for each triangle (via cross product) anyway. But, if you require properly calculated normals in your STL, set it to false: "no_normals":false
  • ignore_leq: default: null. Using an elevation (e.g. 0.0), this  will ignore any cells less or equal to that elevation. Good for omitting offshore cells and print only onshore terrain. Note that 0 may not be exactly sealevel on some DEMs, you may have to experiment with slightly higher values to remove all of the water, e.g. 0.5: "ignore_leq":0.5
  • projectiondefault: null. By default, the DEM is reprojected to the UTM zone (datum: WGS84) the model center falls into. The EPSG code of that UTM projection is shown in the log file, e.g. UTM 13 N, EPSG:32613. If a number(!) is given for this projection setting, the system will request the Earth Engine DEM to be reprojected into it. For example, maybe your data spans 2 UTM zones (13 and 14) and you want UTM 14 to be used, so you set projection to 32614. Or maybe you need to use UTM 13 with NAD83 instead of WGS84, so you use 26913. For continent-size models, WGS84 Web Mercator (EPSG 3857), my work better than UTM. See https://spatialreference.org for descriptions of EPSG codes. Be aware, however, that Earth Engine does not support all possible EPSG codes. For example, North America Lambert Conformal Conic (EPSG 102009) is not supported and gives an error message: The CRS of a map projection could not be parsed
  • only: Giving two tile index numbers [x,y], this will only process one tile and ignore all other tiles. [1,1] is the upper left tile, [2,1] it the tile to it's right, etc. This will enable users to d/l otherwise unreasonably large models by processing one of its tiles at a time (thus staying under the server limit), then manually selecting the next tile and process it, etc.

    Example:  You have 4 tiles, each 250 mm wide, so you can't process all 4 of them at once b/c of the server limits. Add the option "only":[1,1] and make sure to Save to URL before hitting Export. This will only process the tile with index 1,1. Once this tile was downloaded, go back to the main screen and change it to "only":[2,1], repeat for all 4 tiles. 
    Note that each tile will
    be in a new zip, so you should unzip all of them in a common folder. In a 3D viewer, the tiles will fit together without overlaps if tile_centered was false.





Thursday, March 7, 2019

State of TouchTerrain as of version 1.21

State of TouchTerrain 

Chris Harding, Sep. 18, 2018


Tell us about your project

Despite its relative simplicity TouchTerrain continues to grow in popularity since it went live in March 2017. At least that's what Google Analytics tells me. Besides that, we've heard from a number of people using it for cool projects ... but we only find out about those projects when something is broken :)

While that feedback is vital to improve (or fix) things, we would be delighted to hear from any cool project you're running, just so we can feel better (and also to have some material for justifying the ongoing development, slow it may be ...). So, if you're using the 3D models for say academic research, or teaching visually impaired students or presentations in a museum, and you feel inclined to say thanks to us, drop us a note.

Development summary (up to v.1.21)

Here's a quick summary of what I fixed/added since version 1.14
  1. Added region from kml file: The Web app has a Set Box from polygon in KML file button. Digitize an area as a polygon (some sort of box) in Google Earth and save it as a kml file (not kmz). Use the button to upload your kml file and the app will fill the top right and bottom left coordinates with a rectangle surrounding your polygon and fly you to this area.
  2.  Fixed aliasing artifacts (web and stand-alone): At least two users brought this issue to my attention, which manifested as subtle line patterns across the terrain. Even when noticeable in a 3D viewer,  usually they would not show up in the print (due to the 3D printing process). I guessed (correctly, as it turned out) that it was a resampling artifact but was initially not sure where in the process it was introduced. After some poking around I found that Google Earth Engine, which I tell to downsample and project the raster before downloading it for processing, is using nearest-neightbor resampling by default, which leads to these artifacts. Luckily, it's easy to tell it to use a better algorithm, either bi-cubic or bi-linear (which I'm using now). With this, the artifacts seem to have gone.
     
  3. Use full resolution (printres = -1) (stand-alone version with local raster files only): Unlike the web version, using local raster files with the stand-alone version does not yet have an easy way to use bi-linear interpolation. The current method,  uses some numpy trickery to performs the equivalent to nearest-neightbor resampling, again with the potential for aliasing artifacts. Until I find time to add better resampling I've added a way to create 3D models from the resolution of the raster file, i.e. with no down-sampling. This won't show any artifacts but may result in very large meshes.
  4. Added DEM to zipped folder: The geotiff I downlaod from Google Earth Engine is now included in the zipped folder you download from TouchTerrain, it's always called DEM.tif.


Finally a FYI:  you may encounter a dialog box with Google telling you "This page can't load Google Maps correctly". This happens because our server has been visited by too many users and Google recently (July 2018), requires that we pay them for usage over this limit. To be clear, this will only happen if we've exceeded the allowed free quota for the day (I think) and I've encountered it only once, late a night. If you get this dialog, press OK and move on. For our app, even the "ugly" version of the streetmap is usable for finding the region you want to print. The hillshade map is not affected by this.