The View Up Here

Random scribblings about kites, photography, machining, and anything else

Photosynth from a Kite

Posted by Tom Benedict on 12/06/2009

Photosynth is a web-based program that lets you assemble a number of pictures of a single subject and combine them into a 3D navigable scene built from your pictures.  It doesn’t create a 3D model, but it does arrange all the still pictures so that a viewer can move around the scene and see each picture from the angle it was taken at.  People have made some truly phenomenal synths of a range of subjects.  It’s well worth a visit.

Of course I had to try it from a kite.  I pulled out the raw pictures from a couple of previous KAP outings and came up with some decent synths:

As it turns out KAP is very well-suited to making synths.  To make a well-connected synth requires a number of pictures, capturing the subject from multiple angles.  By its very nature KAP takes pictures from a variety of angles and positions.  I’ve made a number of KAP flights specifically to generate a synth, but for the most part they have been the serendipitous outcome of a flight otherwise geared toward still photography.  These are some of the better ones, taken with Photosynth in mind:

But there are times when things just plain don’t work.  Photosynth doesn’t like water because picture-to-picture, water varies, so there’s nothing really to tie one image to the next.  I’ve had some oddball results from synths that included large bodies of water.  Special techniques need to be used.

One in particular, a synth from a set of flights over the Kiholo Bay Inlet, had a number of problems with it.  Discussion with the Photosynth development team led to the conclusion that it was the water, the lack of a good pool of overlapping images, and just a poor sampling of the area in general that led to the problems.  I’m planning a second trip to Kiholo Bay to try to take a better set of pictures for making a synth there.

An added side-benefit of Photosynth is that in the process of tying all the images together, it creates a point cloud indicating which points in the images tie to points in other images.  A good set of high detail pictures can generate a very dense point cloud.  It’s possible to extract the point cloud and use it to create a rendered 3D image of the scene.

Photosynth Surface Extraction

This model was generated using the point cloud from the Waikoloa Archaeological Site in the list above.  Considering there was no GIS data, no ground control grid, no real spatial or metric information of any kind, it’s remarkably accurate.  Though it also shows some of the problems with this method.  The road surface in the cut is quite smooth, so there are patches without any points in the point cloud.  These show up as gaps in the surface.  There are also some very rough areas of terrain on the same size scale as the mesh spacing in the surface.  This caused issues as well.  All in all, though, it’s a neat technique.

But the flight at the Kiholo Bay Inlet has been the most instructive of all.  Because of the sparse data set, it really pushed the Photosynth algorithm, and indicated some new approaches that would help with making synths from a kite.  When I go back, the plan is to:

  • Get the camera airborne over the water, and walk the length of the pond while pointing it at the far shore.  Take pictures every five seconds to get good overlap between frames.
  • Turn the camera 180 degrees and walk back the other way, taking pictures of the near shore.  Again, take pictures every five seconds to get good overlap between frames.
  • Turn the camera 90 degrees to face down the length of the pond, and tilt it down until the horizon is just out of view.  Walk the length of the pond taking pictures every five seconds.
  • Turn the camera 180 degrees to face back the other way down the length of the pond, and walk back taking pictures every five seconds.

At this point there should be enough images with enough overlap to make very good references for both shores, and enough images to tie the two shores together into a single frame of reference.  From here on out images can be taken to place them inside this frame of reference:

  • Get the camera to a good high altitude over the far shore, 50-100m or so, and begin taking hemispherical panorama sets.  Start at one end of the pond and work toward the far end, moving maybe 10-20m between sets.  This should provide a view in every direction from any point above the pond, and the high altitude should let the straight-down ortho shots tie into the frame of reference we generated above.
  • Make a second pass at a lower altitude with the camera over the near shore.  Similar spacing between hemispherical sets.
  • Finally, switch back to manual control and take detail sets of some of the features at the site (resting turtles, the turtle observation station, various rock features, key features along the shoreline, etc.)

At the end of the day, the pictures need to be culled to remove:

  • Blurries – Photosynth doesn’t work well with them, and they’re no fun to look at.
  • Bad exposures – For the same reasons as above.
  • Frames with more than 50% water in them – This was one of the key issues with my first attempt at this site.  Photosynth doesn’t deal well with water.

Finally, put all the images into Photosynth and see what comes out!

It’s a long process, and the number of images involved can be quite large.  Each set of images for the hemispehrical panoramas is 40 to 48 frames.  Ten spots along the pond can generate almost 500 images.  The two passes will crank that number up to about a thousand.  Throw in the initial framework images, and the total will probably exceed 1500 images.  This is more than I did at Green Sand Beach, by far my largest synth to date, but it should make for a very complete synth of the Kiholo Bay Inlet.

I should have a chance to try this over the weekend, if the weather holds.  Time will tell.

Tom

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: