Wednesday, June 12, 2019

TouchTerrain 2.0-beta is out!

Since Fall last year, I've been working on improvements and fixes  In some cases, I've already added them to the current version (1.2x) but most of them are new ... and untested by real users.  I therefore decided to have a beta phase with a separate beta server, as to not interrupt the old server. 

To try the beta version got to:

Any feedback is very much appreciated, please use to email us any bugs, observations, requests, etc. Please note the version of the beta (see title, will be something like beta-0614) and add screenshots if needed. Thanks!

Finally, huge shoutout to Levi and Nick from ISU Research IT for setting up a docker-webhook-magic-thingy that should (theoretically) enable me to make fixes/changes to my code and update the server directly myself! This should results in much faster turnaround on fixes to the beta.

What's new in 2.0?

Despite not having gotten much of a face-lift, TouchTerrain 2.0 has a lot of new features behind the scenes! I also re-visited some areas that needed more attention and improved or at least cleaned up some issues.

3D Preview: Once your model has been processed, you can preview it in 3D (rotate, zoom, etc.) right in your browser. This uses's Javascript plugin which uses Three,js (WebGL) for rendering. As everything happens inside your browser, it may take a while to actually show you the graphics, especially for large models.

Snazzy wait-animation: I still can't show you how far your processing has gone, but at least now you get a nice animation while you wait. Also, there's no preflight screen anymore, as soon as you hit Export, processing will commence. It's still recommended to hit Save to URL before you hit Export, otherwise your current settings won't get save to the URL and you might need them later!

Geotiffs: you will now always get a geotiff of the area that you requested (red box) but projected into UTM at the resolution used to created the STL(s). E.g. if you requested a print resolution of 0.4 mm and this translated into say 27.6 m for each cell, this will be the resolution of the geotiff. If you requested several tiles, the geotiff will be the pre-tiled version i.e. you'll only get one geotiff, not many.

Geotiff as File Format: This will only give you the Geotiff as output. Use this with print resolution set to source resolution to grab the geotiff at its native resolution. This should be very handy for users who just want a geotiff (e.g. for GIS work) but need the best possible resolution.

Print resolution (printres) set to source resolution: every DEM source has a "native" resolution (e.g. ~10 m for NED). Using source resolution for the print resolution instead of a mm number, will use this native resolution and give you the highest possible level of detail for your 3D terrain model. 
This is primarily  meant for the geotiff-only output, where it ensures that no interpolation due to re-sampleing is done. In other words, you'll pretty much get the original data except it will be projected to UTM. E.g. for NED it'll be the same data as you'd get from the USGS TNM website and you don't have to d/l a full 1 degree block and clip out your area.
For STL/OBJ, using the source resolution will result in the highest detail possible 3D models albeit at the cost of very large files, so you'll have to be content with relatively small areas or run up against our server limits. (Note: if you run your own server or use the stand-alone version, you can go nuts with this, just don't be surprised if processing takes a long time and gives you huge files :)

More precise model dimensions: I spend some time making sure that the area you select and the model you actually get are the same. The tricky part is that the red selection box provides four corners, each given in lat/long. However, the lat/long box is always more or less a trapezoid, not a rectangle (thanks, "round" Earth!) but the geotiff I get from EarthEngine is in UTM (meter based) and is always a rectangle.
I did work to improve this by adjusting the print resolution a tiny bit if needed. Previously, a printres of 0.4 might have resulted in a raster is 234.576 cells wide, but as there are no partial cells it had to be shortened to 234 cell and thus the model was a bit too small. Now, I adjust the printres (to say 0.3994554) so that the resulting raster exactly 235 cells wide. This also improves the fit of tiles. Previously it was possible that the tiles were a bit too short and when you assembled them you could see small offset artifacts were they touched. 
The log file will show when such adjustments were made and what your defacto print resolution and model dimensions are, e.g.:

cell size: 109.05937041208486 m  
adjusted print res from the requested 0.5 mm to 0.4784688995215311 mm to ensure correct model dimensions 
total model size in mm: 100.0 x 99.52153110047847 
Cropping for nice fit of 2 (width) x 2 (height) tiles, removing: 1 columns, 0 rows 
 cropped (209, 208) to (208, 208) 
 cropping changed physical size from 50.0 mm x 49.760765550239235 mm 
 to 49.760765550239235 mm x 49.760765550239235 mm 
map scale is 1 : 227934.08416125734 

Server Limits: We're still tuning those but the current limits should allow models of up to 300 x 300 mm at 0.4 mm resolution (or 3 x 3 tiles of 100 x 100 mm each, etc.). This should work for areas at the equator. As the farther you get towards a pole, the less data you're actually requesting, you should actually be able to get even larger models away from the equator. If you're over the limit, you will be told by how much, so you can go back and adjust your area. We think that's a pretty generous model size, given that most common  3D printers don't even have 300 x 300 mm buildplates (CR-10s, etc. non-withstanding).  If you need multiple tiles of that size, there's a "cheat" you can use (see below).

Which brings us to the manual options:

Manual options: There's now a large field beneath file format to manually put in options and values. Some options are not part of the UI options , but you can override UI options! To put in an option and an override value, type in the name of the option in double quotes, then a colon, then the value (no spaces). If you want to add other options, add a comma and space and repeat. The name of an UI option is shown when you mouse over the UI part: it's the first word in parens.

  • You want a print resolution of 0.37 mm, not 0.4 or 0.35 
  • The name of that options is printres, so you type in:
  • "printres":0.37
  • You also want a model width of 175 mm, name for this is tilewidth, so you add that:
  • "printres":0.37, "tilewidth":175

 Now for the fun part, where we use the new, non-UI options:

  • tile_centered: default: false. With false, all tiles are offset so they all "fit together" when they all are loaded into a 3D viewer, such as Meshmixer or Meshlab. Note that slicers (e.g. Cura) have ways to put the model into the center of the build plate, even if its center coordinates is not 0,0. Set this to true to force all tile STL to be centered around 0/0: "tile_centered":true
  • no_bottom: default: false. Will omit any bottom triangles i.e. only stores the top surface and the "walls". This creates ~50% smaller STL/OBJ files. When sliced it should still create a solid printed bottom (tested in Cura). This option is false by default. meaning you will get all the bottom triangles unless you overwrite it with true:  "no_bottom":true
  • no_normals: default: true. Will NOT calculate normals for triangles in STL files and instead set them to 0,0,0. This is significantly faster and should not matter as on import most slicers and 3D viewers will calculate the normal for each triangle (via cross product) anyway. But, if you require properly calculated normals in your STL, set it to false: "no_normals":false
  • ignore_leq: default: null. Using an elevation (e.g. 0.0), this  will ignore any cells less or equal to that elevation. Good for omitting offshore cells and print only onshore terrain. Note that 0 may not be exactly sealevel on some DEMs, you may have to experiment with slightly higher values to remove all of the water, e.g. 0.5: "ignore_leq":0.5
  • projectiondefault: null. By default, the DEM is reprojected to the UTM zone (datum: WGS84) the model center falls into. The EPSG code of that UTM projection is shown in the log file, e.g. UTM 13 N, EPSG:32613. If a number(!) is given for this projection setting, the system will request the Earth Engine DEM to be reprojected into it. For example, maybe your data spans 2 UTM zones (13 and 14) and you want UTM 14 to be used, so you set projection to 32614. Or maybe you need to use UTM 13 with NAD83 instead of WGS84, so you use 26913. For continent-size models, WGS84 Web Mercator (EPSG 3857), my work better than UTM. See for descriptions of EPSG codes. Be aware, however, that Earth Engine does not support all possible EPSG codes. For example, North America Lambert Conformal Conic (EPSG 102009) is not supported and gives an error message: The CRS of a map projection could not be parsed
  • only: Giving two tile index numbers [x,y], this will only process one tile and ignore all other tiles. [1,1] is the upper left tile, [2,1] it the tile to it's right, etc. This will enable users to d/l otherwise unreasonably large models by processing one of its tiles at a time (thus staying under the server limit), then manually selecting the next tile and process it, etc.

    Example:  You have 4 tiles, each 250 mm wide, so you can't process all 4 of them at once b/c of the server limits. Add the option "only":[1,1] and make sure to Save to URL before hitting Export. This will only process the tile with index 1,1. Once this tile was downloaded, go back to the main screen and change it to "only":[2,1], repeat for all 4 tiles. 
    Note that each tile will
    be in a new zip, so you should unzip all of them in a common folder. In a 3D viewer, the tiles will fit together without overlaps if tile_centered was false.

Thursday, March 7, 2019

State of TouchTerrain as of version 1.21

State of TouchTerrain 

Chris Harding, Sep. 18, 2018

Tell us about your project

Despite its relative simplicity TouchTerrain continues to grow in popularity since it went live in March 2017. At least that's what Google Analytics tells me. Besides that, we've heard from a number of people using it for cool projects ... but we only find out about those projects when something is broken :)

While that feedback is vital to improve (or fix) things, we would be delighted to hear from any cool project you're running, just so we can feel better (and also to have some material for justifying the ongoing development, slow it may be ...). So, if you're using the 3D models for say academic research, or teaching visually impaired students or presentations in a museum, and you feel inclined to say thanks to us, drop us a note.

Development summary (up to v.1.21)

Here's a quick summary of what I fixed/added since version 1.14
  1. Added region from kml file: The Web app has a Set Box from polygon in KML file button. Digitize an area as a polygon (some sort of box) in Google Earth and save it as a kml file (not kmz). Use the button to upload your kml file and the app will fill the top right and bottom left coordinates with a rectangle surrounding your polygon and fly you to this area.
  2.  Fixed aliasing artifacts (web and stand-alone): At least two users brought this issue to my attention, which manifested as subtle line patterns across the terrain. Even when noticeable in a 3D viewer,  usually they would not show up in the print (due to the 3D printing process). I guessed (correctly, as it turned out) that it was a resampling artifact but was initially not sure where in the process it was introduced. After some poking around I found that Google Earth Engine, which I tell to downsample and project the raster before downloading it for processing, is using nearest-neightbor resampling by default, which leads to these artifacts. Luckily, it's easy to tell it to use a better algorithm, either bi-cubic or bi-linear (which I'm using now). With this, the artifacts seem to have gone.
  3. Use full resolution (printres = -1) (stand-alone version with local raster files only): Unlike the web version, using local raster files with the stand-alone version does not yet have an easy way to use bi-linear interpolation. The current method,  uses some numpy trickery to performs the equivalent to nearest-neightbor resampling, again with the potential for aliasing artifacts. Until I find time to add better resampling I've added a way to create 3D models from the resolution of the raster file, i.e. with no down-sampling. This won't show any artifacts but may result in very large meshes.
  4. Added DEM to zipped folder: The geotiff I downlaod from Google Earth Engine is now included in the zipped folder you download from TouchTerrain, it's always called DEM.tif.

Finally a FYI:  you may encounter a dialog box with Google telling you "This page can't load Google Maps correctly". This happens because our server has been visited by too many users and Google recently (July 2018), requires that we pay them for usage over this limit. To be clear, this will only happen if we've exceeded the allowed free quota for the day (I think) and I've encountered it only once, late a night. If you get this dialog, press OK and move on. For our app, even the "ugly" version of the streetmap is usable for finding the region you want to print. The hillshade map is not affected by this.

Friday, November 3, 2017

TouchTerrain version 1.14 is out

Web app changes:

  • v1.14 fixed the Server returned HTTP code: 404 error. 
  • Levi and Shane also changed the apache config settings which should make the  'NoneType' object has no attribute 'route'  error less likely.  Let us know if its still happening so we can learn more about it.
  • The Internal Server error also seem to have gone. (Report if you get it)

Stand alone version changes (see  Github repository)

  • Instead of a web interface, the Stand alone version gets all its parameters (area coordinates, DEM source,  3D print settings) via a JSON file. To get DEM data from Google Earth Engine (like the web app does), you need to set up a GEE account.
  • v1.14 adds the option to use your own DEM raster file (geotiff, img, etc.) instead. In this case, no Google Earth Engine is required!
  • Here's an example JSON file that processes a geotiff called pyramid.tif (my test file):
  • "CPU_cores_to_use": 1, 
    "DEM_name": "USGS/NED", 
    "basethick": 2,  
    "fileformat": "STLb", 
    "importedDEM": "pyramid.tif", 
    "ntilesx": 1, 
    "ntilesy": 1, 
    "printres": 1, 
    "tile_centered": true, 
    "tilewidth": 120,  
    "zip_file_name": "pyramid", 
    "zscale": 0.5
  • If there's interest, we could add this to the web version as well, so you could upload your own DEM file. 

Friday, October 27, 2017

Errors with the current version (v.1.09): 

We are aware of the following errors and are working to fix them. Python error message after pressing Start: Server returned HTTP code: 404 or 'NoneType' object has no attribute 'route'  or   (after a while):  Internal Server Error

Sometimes going back and trying again may make it work but it's clearly super annoying.

If you want to be notified after we've fixed it, leave a comment.

If you desperately  need a model now contact us as and we can probably create it for you. 

Thanks for your patience!