Friday, April 30, 2021

What's new in TouchTerrain version 3.3?

  • Help "bubbles": When you mouse-over or click on a question mark icon, a help text (popover) will appear. If you clicked on the question mark, the popover will remain visible until you click on it again.

  • 3D printer option settings for CNC users: This is mean to help the around 15% of TouchTerrain users who create terrain models using CNC machines. There's no direct equivalent to physical print size or nozzle nozzle, diameter,  so for CNC users, I've added three presets: small, medium, large size, which is related to the final, physical size of the piece of wood; and low, medium and high detail, which is related how much detail your carve can potentially contain.
  • Medium offers a reasonable compromise of level of detail and the size of the STL. If you want a smaller STL and can deal with less detail, set this to low detail. If you are sure that you need more detail, select high, which will create larger STL files and take longer to process. Preview can help to decide on the best level of detail but, as many factors influence the final outcome, you'll have to experiment with different settings in your CNC software and see which work best
  • For CNC users, the rest of the 3D printer settings can be left at their default level. Be aware that using a >1x z-scale can help to make low-relief terrain more interesting. However, this can also be done later in your CNC software.

  • Auto-height values for z-scaling: Many users who leave the z-scale value at the default 1x may be missing out 3D terrain prints that are more visually appealing, especially in low-relief areas. It good practice to set a z-scale that end up creating a 3D print with at least 50 layers, as less layers will not be able to image small terrain details, such as meandering river beds. 
  • However, it can be a pain to determine the "best" z-scale, which in TouchTerrain can really only be done through trial and error using the Preview, (It is much easier to increase the z-scale in the slicer software until it looks interesting!). I've added a few presets for the desired height of the model (i.e. the physical distance from the lowest elevation to the highest) in either mm or inches. 
  • A height of 20 mm is good starting value. Preview the model and change the height as desired. The actually used z-scale will be given in the log file.

  • kmz files for polygons: You can now use kmz and kml files to provide polygons to define your print area's boundary

Wednesday, March 31, 2021

TouchTerrain Story Map

I've made my first ever ESRI Story map about TouchTerrain: It re-uses some of the material from the AGU 2020 conference poster but is far slicker!

Tuesday, March 30, 2021

Version 3.2.1: Z-scale bug fix

Z-scale fix 

  • Well, this is embarrassing! Simon Scherrer recently pointed out that my mesh models where hilariously tall when applying a large z-scale. 
  • Turns out that at one point I had done the z-scaling in one function, then forgot about it and applied it again in another function! 
  • While easily fixed, this means that any z-scale larger than 1.0 was wrong (not sure since when, must be several years ...). For most users that's likely not an issue b/c it wasn't that obvious at low (>1.0) z-scales and many (including) myself have no real world experience with how a certain z-scale should properly  look like. (This is what convinced me that my old z-scale of 5 was WAY too much! Also, I like that they call the z-scale Boost factor!)
  • Also, I think most user would have dialed in a appropriate z-scale  based on visual inspection. However, if you've been using TouchTerrain for anything scientific where proper z-scale matters you should account for my error and either correct your effective z-scale to the square of what you requested pre version 3.2.1 or re-do you model with the new (correct) z-scale method now in place. 
Old (pre version 3.2.1) z-scale of 5 aka Mordor mode!

Hillshaded relief with proper z-scale of 5

Fixed (>= version 3.2.1) z-scale of 5

  • Again, thanks to Simon Scherrer for bringing this to my attention!

Friday, March 12, 2021

What's new in TouchTerrain Version 3.2?

Docker version of TouchTerrain

  • I created a Docker image of TouchTerrain that makes is easy(er) to run the stand-alone version via a jupyter notebook.
  • After installing the image and running the container, you will have a virtual Linux box running inside your PC/Mac, with all required python packages already installed.
  • Shell scripts are provided to install the latest version of TouchTerrain and to run a jupyter server inside the container. The server that can be accessed through your standard, local browser and you can download and upload files through jupyter.
  • A new notebook (TouchTerrain_jupyter_for_starters.ipynb) will hopefully make is easy for beginners to work through the workflow of creating a 3D terrain model file in standalone mode.
  • The github repo for the Docker version is: 

Added two new DEM sources:

  • NRCan/CDEM: The Canadian Digital Elevation Model (CDEM) is part of Natural Resources Canada's (NRCan) altimetry system and stems from the existing Canadian Digital Elevation Data (CDED). The CDEM is comprised of multiple DEMs with varying resolutions. These vary according to latitude and have a base resolution of 0.75 arc-seconds (about 20m).

  • AU/GA/AUSTRALIA_5M_DEM: The Digital Elevation Model (DEM) 5 Metre Grid of Australia derived from LiDAR model represents a National 5 metre (bare earth) DEM which has been derived from some 236 individual LiDAR surveys between 2001 and 2015 covering an area in excess of 245,000 square kilometres. All available 1 metre resolution LiDAR-derived DEMs have been compiled and resampled using a neighbourhood-mean method to 5 metre resolution.

Note that both DEMs are image collections, rather than a single image as can seen here for the Australia DEM (black patches):

Zoomed into an urban area. As Lidar cannot image water surfaces, these areas will always be set to NaN and will be omitted from the terrain model.

Even at "only" 5 m resolution, it still shows some nice details!

CDEM coverage:

Saturday, November 21, 2020

What's new in version 3.1?

Optimized Wall layout for polygon-based models

For 3.1, I changed the way walls are set up if the models was created via a polygon. This means that the raster contains NoData (NaN) cells. Cells containing a elevation were each simply expressed as a quad,  with a top and bottom and possibly with one or more walls. This simple rasterization method worked but lead to a gazillion "corner quads" (i.e. quads with 2 or 3 walls). And while these corners are typically nearly imperceptible in the final 3D print, I found that it was very hard on the motion system of my 3D printer to rattle along all those tiny teeth and it also took longer to print.

Old method

My solution is to "snip off" half of any quad that has 2 adjacent walls and replace them with a single diagonal wall:

New method


Note that I did not optimize quads with 3 adjacent wall b/c it would have to either randomly choose one of the two possible diagonals or do something drastic like removing this quad or replace it with some sort of "nose".

Here's an example of a non-trivial case created with this optimized wall strategy:


Added the MERIT 90m DEM source

It's only 90m resolution but is more precise, so maybe it's useful for those printing river networks and other flat areas (like Iowa :)

MERIT DEM a high accuracy global DEM at 3 arc second resolution (~90 m at the equator) produced by eliminating major error components from existing DEMs (NASA SRTM3 DEM, JAXA AW3D DEM, Viewfinder Panoramas DEM). MERIT DEM separates absolute bias, stripe noise, speckle noise and tree height bias using multiple satellite datasets and filtering techniques. After the error removal, land areas mapped with 2 m or better vertical accuracy were increased from 39% to 58%. Significant improvements were found in flat regions where height errors larger than topography variability, and landscapes such as river networks and hill-valley structures became clearly represented.

Another potential benefit is that MERIT is not defined for offshore locations, making it convenient for clean models of coastal areas, where you may not want a "flat ocean"

Using the AW3D30 DEM:
Using the MERIT DEM:

Friday, October 9, 2020

TouchTerrain 3.0: New GUI, place search, kml polygon file upload

What's new in Version 3? 

New GUI: With version 3.0 (October 2020), TouchTerrain has gotten a serious facelift. Gone is the utilitarian ca. 1995 aesthetic, replaced by a Bootcamp 4 based GUI. This is my first ever dip into using bootcamp and there are probably still some details to be worked out. 

GUI responsiveness: The Bootcamp GUI implements what it calls responsiveness which means that (to an extent) the app will try to dynamically"compact" itself when used with small screens (tablets, smartphones). Note that this should work on a smartphone but will look a bit weird. Hopefully most users will have a reasonably wide display when using the app!

GUI size adjustments: You will notice that there are 3 main parts ("cards" in Bootstrap speak) to the right of the map. Clicking on their titles (e.g. on Area Selection Box:) will grow or shrink them. By default, only the Terrain Settings and the 3D Printer Options parts are "open", the Area Selection Box (which hides the lat/long coordinates of the box corners) is closed. Opening all three together will make the green Export Button on the bottom to be shoved out of sight, so you'll have to close a part or scroll down to click on it.

Manual box coordinate entry: In version 3, the four coordinate fields inside Area Selection Box are now active. Changing any of the numbers and hitting Enter will immediately resize the red box accordingly.

"Oversampling" warning: If you configure things in a way that TouchTerrain will interpolate the DEM into a higher (more detailed) resolution that what the DEM source can provide, the Current DEM resolution value will turn yellow. Ex: The source DEM resolution is 30 m but you're requesting 20 m. Using 20 m will still "work" but it's counterproductive b/c it won't magically create more detail and your files will be needlessly larger than if you had configured it to resample at say 32 m instead.

Place search bar: As is standard in many other Google Map applications, you can now search for a place by typing a search term into the box and pick one of the five suggestions displayed.  Your map view will fly to the result you picked. Be aware that it will NOT move the red selection box to your new view, so don't forget to hit the blue Re-center box button when needed!

Use a Polygon from a kml file: If you have a kml file that contains a polygon or a polyline (e.g. the outline of a state) you can upload it to the app and use it as boundaries for the 3D model you download. 

To upload the kml file, click on Browse, select a kml file from you local drive and hit Open. Note that is has to be kml file, not a kmz file! Kmz is just the zipped version of kml. To use a kmz, unzip the kmz file, which will give you a file called doc.kml, just rename that and then upload it. You could of course also use Google Earth to digitize a polygon, save it into a kml file and upload it to the app.

After the upload your map view will jump to the area and will see your polygon in yellow inside the red box. Adjust the other settings and then hit Export. You will see a note (Using X points from kml file as polygon) and you model (and geotiff file) will be clipped (masked) by the polygon.

Note that this polygon will be cleared if you move the red box or if you use functionality that requires the app to reload, such as changing the DEM source or the hillshade settings. This is needed, b/c the polygon isn't inlined into the URL (like all the other settings). But, it should be pretty quick to re-upload the kml file again! Finally, be aware that if your file contains multiple polygons or polylines, only the first will be taken.

Other changes:

  • URL: Depending on you system you may no longer see the full URL in the browser. However, we now give you the full URL at the end of the export (as simple browser text), so you can still use it to "give" your model to somebody else.
  • GPX path lines:  On the standalone version of TouchTerrain, you can now drape path lines from gpx files over your model.
  • Map with jupyter notebook: The jupyter notebook standalone version can now show you an interactive Google Map in a notebook cell (using the geemap module). It also lets you digitize a rectangle, circle or polygon onto that map and use it as your model area.

Wednesday, May 13, 2020

touchterrain 2.5: New Hillshading, more DEM sources, lower_leq setting and new folder structure (May 13)

The May 13 release of touchterrain (v. 2.5) contains a whole bunch of new and changed features:

New Hillshading

  • Hillshading creates a overlay map that shows a pseudo-3D relief of the underlying DEM. This is useful as a pre-visualization of how your 3D print will roughly look and helps users to select interesting areas to print.
  • Hillshading (in its simplest form) assumes that a sun is positioned above the terrain at certain angles:
    • Azimuth: Defines the direction (compass heading) from which the Sun shines to create the hillshade relief. Note that South will typically result in a flipped relief perception!
    • Elevation: Defines the vertical Sun angle, i.e. how high the Sun is above the horizon. Use it to enhance detail for different types of terrain.
  • So basically we have a 2 angles, a horizontal (0 - 360, 0 = North) and vertical (0 - 90, 90 = perfectly above) that define the position of the light source used to create the impression of a 3D relief by creating greyscale values from 0 (dark) to 255 (super bright). Note that this does not create shadows, only the effect of specular highlights, which is a very strong perceptional cues for creating a 3D effect.
  • The standard hillshade parameter values are: azimuth of 315° (NNE) and 45° for elevation, which are good all-around settings.
  • However, it can be fun (and rewarding) to change these angles, which is what the new version allows you to do (in a limited fashion)!
  • For example, for best results the Sun should come at an angle roughly perpendicular to ridges in your terrain. Here's a comparison for different azimuth values (elevation angle is 60) (source):

  • 270, 315, 0 and 45 all bring out different aspects of the same data, e.g. 45 emphasizes NE-SW oriented features. Also, to me, some values seem to make my brain reverse the topography i.e. ridges look like valleys and vice versa. 
  • Looking at the elevation value, values lower than 45, create darker and darker overlays but also more and more emphasize low relief terrain features, such as drainage channels (but at the cost of blowing out high relief features).
  • For touchterrain 2.5 I have attempted to implement a simple way to experiment with these two hillshade angles without overwhelming and confusing the user:
  • Sun direction (new) has a few direction presets shown as words (West, North-West, etc.). I only sweep the northern directions in 45 steps b/c I'm pretty sure that using southern direction would result in relief confusion. But, just to force the effect I include South as well.
  • I renamed the Elevation parameter to Sun Angle (b/c I want to avoid confusion with terrain elevation) and again offer just a few presets: Normal (45), Steep (55) and several progressively flatter settings to go below 45 all the way to 5
  • Finally there's a gamma value via a direct number input. This gamma correction factor is used to somewhat compensate that low sun angles tend to create hillshades that look too dark.  Gamma values > 1 increase the "lightness" so as the sun angle becomes lower, this gamma will increase to compensate. Note that the specific gamma values are my "opinion" only! If you want to tweak the gamma after selecting a sun angle preset, just change the number and hit enter. Feel free to experiment, but be aware that your tweak will be overwritten next time you choose another sun angle preset!
  • Be aware that changing any of the hillshade parameters will require a reload of the map, b/c I need to give Earth Engine these new values, get back a new hillshade overlay calculated for the new values and then display it. So it might take a split second, depending on your internet.
  • Note that I have tried to preserve all other input values you might have chosen before doing such a reload. This includes the type of Google Map map in the background and the transparency. However, this is not fully tested, so please let me know if values for some fields you changed earlier are getting reset.
  • If you're curious, go and change the Google Map to Terrain (Map -> Terrain), which also has hillshading backed in, and lower your transparency to mix them both.
  • Transparency (inverse of opacity) is still given in percent but I made the slider much, much smoother (1% steps). I also removed the direct numeric input b/c I feel having a precise number value is not how transparency is used, user should simple move the slider until they are happy.  Changing Transparency will not trigger a reload, b/c it's set via your browser (client) not via the server.
  • Finally, there are a bunch of new query parameters in the URL that store the new hillshade settings and transparency: transp= 0 -100,  hsazi= -360 - 360 (hs azimuth), hselev=  0 - 90 (hs elevation) and gamma= 0 - 99, so if you need values that cannot be generated in the GUI, just change those in the URL and hit enter. Note that none of these parameters matter when processing the terrain into 3D models, they are purely for a better visual experience.
  • So ... got nuts and play with these values!  Set the transparency to 0 (full left) and set the Sun angle to extremely flat. I think this looks very interesting (or at least artistic :)

More DEM sources

  • I have added a few new sources and added a link to their meta data (via earth engine):
    • USGS/NED: 10 m, continental USA only. link
    • ALOS DSM: Global: 30 m, worldwide, but has some small "holes". link
    • USGS/SRTMGL1_003: 30 m, "worldwide", but not very far north (lower quality and older than ALOS!). link
    • USGS/GMTED2010: ~230 m, truly worldwide. link
    • GTOPO30: 30 Arc-Second, 1000 m, 1996, worldwide. link
    • CryoSat-2 Antarctica: 1000 m, antarctica only. link
    • NOAA/NGDC/ETOPO1: 2000 m, worldwide, with bathymetry. link
  • For any non-US area ALOS should be your preferred choice as it's much newer and of better quality. However, it misses data in a few tiny areas ("holes")
  • Using the Antarctica data is sadly a bit silly at the moment b/c Google maps are way too distorted.

lower_leq setting

  • Thanks to Iden Craven a manual setting was added to lower any elevation values that are lower than a threshold (say <= 0 m, i.e. anything not land) by a millimeter value when printed out.
  • Example "lower_leq":[0, 5] as a manual setting will lower offshore areas by 5 mm.
  • This can emphasize shorelines while still printing area below sealevel. (ignore_leq would completely omit these areas when printed)

 New folder structure

  • I decided to create a proper module for most of the touchterrain code, which required a new folder structure:
    • standalone was remove and the two "main" files, and its jupyter notebook version, TouchTerrain_standalone_jupyter_notebook.ipynb (recommended) are now at the root level.
    • All other code (the common and server folders) was moved into a new folder called touchterrain
    • Before running the stand alone version, a module called touchterrain needs to be build and installed in your system's site-packages folder. This is done via, best by using pip: pip install . <-dot!
    • The jupyter notebook now explains the installation and can even run it inside a cell as a shell command!
    • auxiliary files (examples, etc.) were mover to a new folder called stuff