Monday, September 16, 2019

TouchTerrain 2.0 is out

TouchTerrain version 2.0 is now live on https://touchterrain.geol.iastate.edu. Big thank you to the beta-testers! https://touchterrain-beta.geol.iastate.edu will still remain active but will be my personal development server where I implement new stuff, which may break the server so you should not use it anymore. Speaking of breaking, if you are experiencing issues (page hangs, not download, etc.) please let us know via Geofablab@gmail.com, same with questions and suggestions! If you are reporting a bug please tell us the 4 digit version shown in the title, e.g. (0914).



Summary of changes and addition for version 2.0

(also here on github)

3D Preview: Once your model has been processed, you can preview it in 3D (rotate, zoom, etc.) right in your browser. This uses www.viewstl.com's Javascript plugin which uses Three,js (WebGL) for rendering. As everything happens inside your browser, it may take a while to actually show you the graphics, especially for large models.

Snazzy wait-animation: I still can't show you how far your processing has gone, but at least now you get a nice animation while you wait. Also, there's no preflight screen anymore, as soon as you hit Export, processing will commence. It's still recommended to hit Save to URL before you hit Export, otherwise your current settings won't get save to the URL and you might need them later!

Optional feedback/survey text box: I've added a textbox just above the download button where users can tell us what they use the terrain models for (150 chars or less). This is completely optional (you can just leave the box empty) and no personal info is stored. However, we would appreciate it if you write a quick note, so that we, over time, can get a rough idea what the use cases for 3D printed terrain models are.

Geotiffs: you will now always get a geotiff of the area that you requested (red box) but projected into UTM at the resolution used to created the STL(s). E.g. if you requested a print resolution of 0.4 mm and this translated into say 27.6 m for each cell, this will be the resolution of the geotiff. If you requested several tiles, the geotiff will be the pre-tiled version i.e. you'll only get one geotiff, not many.

Geotiff as File Format: This will only give you the Geotiff as output. Use this with print resolution set to source resolution to grab the geotiff at its native resolution. This should be very handy for users who just want a geotiff (e.g. for GIS work) but need the best possible resolution.

Print resolution (printres) set to source resolution: every DEM source has a "native" resolution (e.g. ~10 m for NED). Using source resolution for the print resolution instead of a mm number, will use this native resolution and give you the highest possible level of detail for your 3D terrain model. 
This is primarily  meant for the geotiff-only output, where it ensures that no interpolation due to re-sampleing is done. In other words, you'll pretty much get the original data except it will be projected to UTM. E.g. for NED it'll be the same data as you'd get from the USGS TNM website and you don't have to d/l a full 1 degree block and clip out your area.
For STL/OBJ, using the source resolution will result in the highest detail possible for the 3D models albeit at the cost of very large files, so you'll have to be content with relatively small areas or run up against our server limits. (Note: if you run your own server or use the stand-alone version, you can go nuts with this, just don't be surprised if processing takes a long time and gives you huge files :)

More precise model dimensions: I spend some time making sure that the area you select and the model you actually get are the same. The tricky part is that the red selection box provides four corners, each given in lat/long. However, the lat/long box is always more or less a trapezoid, not a rectangle (thanks, "round" Earth!) but the geotiff I get from EarthEngine is in UTM (meter based) and is always a rectangle.
I did work to improve this by adjusting the print resolution a tiny bit if needed. Previously, a printres of 0.4 might have resulted in a raster is 234.576 cells wide, but as there are no partial cells it had to be shortened to 234 cell and thus the model was a bit too small. Now, I adjust the printres (to say 0.3994554) so that the resulting raster exactly 235 cells wide. This also improves the fit of tiles. Previously it was possible that the tiles were a bit too short and when you assembled them you could see small offset artifacts were they touched. 
The log file will show when such adjustments were made and what your defacto print resolution and model dimensions are, e.g.:

cell size: 109.05937041208486 m  
adjusted print res from the requested 0.5 mm to 0.4784688995215311 mm to ensure correct model dimensions 
total model size in mm: 100.0 x 99.52153110047847 
Cropping for nice fit of 2 (width) x 2 (height) tiles, removing: 1 columns, 0 rows 
 cropped (209, 208) to (208, 208) 
 cropping changed physical size from 50.0 mm x 49.760765550239235 mm 
 to 49.760765550239235 mm x 49.760765550239235 mm 
map scale is 1 : 227934.08416125734 

Server Limits: We're still tuning those but the current limits should allow models of up to 300 x 300 mm at 0.4 mm resolution (or 3 x 3 tiles of 100 x 100 mm each, etc.). This should work for areas at the equator. As the farther you get towards a pole, the less data you're actually requesting, you should actually be able to get even larger models away from the equator. If you're over the limit, you will be told by how much, so you can go back and adjust your area. We think that's a pretty generous model size, given that most common  3D printers don't even have 300 x 300 mm buildplates (CR-10s, etc. non-withstanding).  If you need multiple tiles of that size, there's a "cheat" you can use (see below).

Which brings us to the manual options:

Manual options: There's now a large field beneath file format to manually put in options and values. Some options are not part of the UI options , but you can override UI options! To put in an option and an override value, type in the name of the option in double quotes, then a colon, then the value (no spaces). If you want to add other options, add a comma and space and repeat. The name of an UI option is shown when you mouse over the UI part: it's the first word in parens.

Example:
  • You want a print resolution of 0.37 mm, not 0.4 or 0.35 
  • The name of that options is printres, so you type in:
  • "printres":0.37
  • You also want a model width of 175 mm, name for this is tilewidth, so you add that:
  • "printres":0.37, "tilewidth":175

 Now for the fun part, where we use the new, non-UI options:

  • tile_centered: default: false. With false, all tiles are offset so they all "fit together" when they all are loaded into a 3D viewer, such as Meshmixer or Meshlab. Note that slicers (e.g. Cura) have ways to put the model into the center of the build plate, even if its center coordinates is not 0,0. Set this to true to force all tile STL to be centered around 0/0: "tile_centered":true
  • no_bottom: default: false. Will omit any bottom triangles i.e. only stores the top surface and the "walls". This creates ~50% smaller STL/OBJ files. When sliced it should still create a solid printed bottom (tested in Cura). This option is false by default. meaning you will get all the bottom triangles unless you overwrite it with true:  "no_bottom":true
  • no_normals: default: true. Will NOT calculate normals for triangles in STL files and instead set them to 0,0,0. This is significantly faster and should not matter as on import most slicers and 3D viewers will calculate the normal for each triangle (via cross product) anyway. But, if you require properly calculated normals in your STL, set it to false: "no_normals":false
  • ignore_leq: default: null. Using an elevation (e.g. 0.0), this  will ignore any cells less or equal to that elevation. Good for omitting offshore cells and print only onshore terrain. Note that 0 may not be exactly sealevel on some DEMs, you may have to experiment with slightly higher values to remove all of the water, e.g. 0.5: "ignore_leq":0.5
  • projectiondefault: null. By default, the DEM is reprojected to the UTM zone (datum: WGS84) the model center falls into. The EPSG code of that UTM projection is shown in the log file, e.g. UTM 13 N, EPSG:32613. If a number(!) is given for this projection setting, the system will request the Earth Engine DEM to be reprojected into it. For example, maybe your data spans 2 UTM zones (13 and 14) and you want UTM 14 to be used, so you set projection to 32614. Or maybe you need to use UTM 13 with NAD83 instead of WGS84, so you use 26913. For continent-size models, WGS84 Web Mercator (EPSG 3857), my work better than UTM. See https://spatialreference.org for descriptions of EPSG codes. Be aware, however, that Earth Engine does not support all possible EPSG codes. For example, North America Lambert Conformal Conic (EPSG 102009) is not supported and gives an error message: The CRS of a map projection could not be parsed
  • only: Giving two tile index numbers [x,y], this will only process one tile and ignore all other tiles. [1,1] is the upper left tile, [2,1] it the tile to it's right, etc. This will enable users to d/l otherwise unreasonably large models by processing one of its tiles at a time (thus staying under the server limit), then manually selecting the next tile and process it, etc.

    Example:  You have 4 tiles, each 250 mm wide, so you can't process all 4 of them at once b/c of the server limits. Add the option "only":[1,1] and make sure to Save to URL before hitting Export. This will only process the tile with index 1,1. Once this tile was downloaded, go back to the main screen and change it to "only":[2,1], repeat for all 4 tiles. 
    Note that each tile will 
    be in a new zip, so you should unzip all of them in a common folder. In a 3D viewer, the tiles will fit together without overlaps if tile_centered was false.

Standalone mode

There are now 2 ways to run TouchTerrain on your own local PC/Mac/Linux box. After you get teh code from github look in the standalone folder There is is a python file called TouchTerrain_standalone.py that you can run via CLI or IDE. It does require a JSON config file as argument, which contains the equivalent of the settings from the web app GUI. However, it might be easier to run the jupyter notebook TouchTerrain_standalone_jupyter_notebook.ipynb and hand edit the config setting in it and then run it (here is a html preview of it). This will also give you a in-browser 3D preview of your model(s).