The View Up Here

Random scribblings about kites, photography, machining, and anything else

  • Flickr Gallery

  • Advertisements

Posts Tagged ‘Photosynth’

Photosynth from a Kite – Take II

Posted by Tom Benedict on 16/06/2009

Last Saturday I returned to Kiholo Bay to make another attempt at taking a set of pictures specifically for making a Photosynth of the area.  Unfortunately the wind was much lighter than I was expecting, so the first hour was spent almost entirely with my rig on the ground.  The 6′ rokkaku I use for the bulk of my light wind flying stayed up the whole time, but the launch of the camera itself was delayed.  The wind did pick up later, however, and I got several hours of flying time in before the sun and the fact that I’d missed lunch by over an hour and a half finally drove me away.

Because of the shorter time table, I didn’t get to do everything I’d planned.  The first four passes, in particular, had to go.  But in the end I think this worked well since I’m not convinced they were necessary, or would even make for a good synth in the end.

Instead, I used a prototype autoKAP controller to take almost all the pictures used in the synth.  The prototype is loosely based on some code I had written for the Atmel ATmega168 microcontroller a few years ago, but it’s being developed for a much better, much more fully-featured controller that’s currently in the works.  Unfortunately I’m not at liberty to speak further on that subject, but it’s moving along.  Soon… soon…

The idea behind it is very similar to the idea behind the Gigapan camera controller:  Orient the camera through a series of tilt and pan angles so that the resulting images have good overlap, and can be stitched together into a large panorama.  Only in the case of Photosynth the idea is to get good overlap between pictures so that the viewer can look in any orientation from that single vantage point.

But this is where the requirements for Photosynth and the requirements for a panorama diverge:  In order to make a good synth, it’s not enough to have good overlap with adjacent images.  Images of the same part of the subject also need to be taken from different vantage points in order to provide parallax.  It’s this that lets Photosynth generate its 3D representation of the scene, and find all the spatial relationships between the individual frames that go into the synth.

In the case of the Kiholo Bay Inlet I started at one end and flew the camera up to a good working height, about 200′ above the ground and almost on the other side of the pond.  Once there, the camera took a hemispherical set of images, three images high and about 12 images around.  As the camera panned around to its starting point, I walked about 50-100′ along the pond and stopped so that another hemispherical set of images could be taken.  This process was repeated until I reached the end of the pond.  About this time the camera’s chip filled up, so it was time to switch.

The second pass was made at a much lower altitude, closer to 75-100′ above the ground.  I also moved back as far as I could toward Kiholo Bay, putting the camera over the near shore.  A similar set of hemispherical images was taken at several locations along the near shore, and at the lower altitude.   And again, about the time I finished at the last position the camera’s chip filled up.

All in all I took over 1200 images this way.  After rejecting images with large percentages of water, the few blurry frames that happened when I was taking in or letting out line, or frames that couldn’t be tied directly to another frame, I was left with about 860 images.  These went into the Photosynth software.  The result can be viewed here:

Kiholo Bay Inlet – Kite Aerial Synth

One surprising outcome of this set of pictures is that it did a better job of covering the area than the set I took the previous week where I was in manual control of the camera and rig.  This is mostly due to the brute-force effectiveness that autoKAP has of taking so many many pictures.  Even as I was going through the pictures and culling out the ones that wouldn’t work well with Photosynth, I was taking notes on which photographs to revisit from the standpoint of traditional still photography.

Paddlers Heading Out

There was a lot more activity at the pond this weekend than last, so I had some opportunities to photograph different groups of people kayaking, paddling, swimming, or in my case, flying a kite.

Wainanali`i Pond Mouth

I also had my first really successful picture of the mouth of the inlet where it connects to the sea.  The clouds and sun cooperated this time, and I was able to get some pictures that did a good job of indicating the depth of the water, the width of the channel, and the reason why tidal flow really doesn’t cause a great deal of mixing in the water.

Honu Swimming I

But my favorite part, by far, was the honu.  And this is where the limitations of autoKAP make it less than ideal.  Because the camera is moving on its own, it’s difficult to do intentional wildlife photography with autoKAP.  I’ve done plenty of serendipitous wildlife photography this way, but without some ability to aim the camera in a particular direction, making pictures of particular subjects is difficult. The handful of honu pictures I made that day were all done with the rig and camera under manual control.

Most animals, sea and land alike, don’t spend a lot of time looking straight up.  Once the kite is in the air and the camera has been lifted to a good working height, for all intents and purposes it’s invisible to creatures on the ground.  In this case the camera was roughly 20′ off the ground, and because of the line angle I was about 40′ from the honu.  As far as it was concerend, I wasn’t there.  I’m often asked if I use some sort of a video feedback system to see what the camera sees.  I’ve been tempted, but so far I haven’t strictly needed it.  A KAPer can develop a sense of where the camera is and what it’s pointing at, even without video feedback.  In this case, once I knew the kite and rig was flying well and wouldn’t do anything unexpected, I positioned the camera above the honu and off to one side, and started taking pictures as it foraged on the rocks.

At some point I’ll return to Kiholo Bay just to take pictures of the honu.  But this trip was all about the synth.  Having it come out as well as it did really put a smile on my face.

Tom

(Note:  I have been calling the Kiholo Bay Inlet by the wrong name: Wainanalii Pond.  This is how it is commonly labeled on maps, hence my mistake.  But during a conversation with Bobby Camara, he pointed out that the village of Wainanalii is a two hour canoe ride to the north of Kiholo Bay, and that it was destroyed by the same lava flow that created the inlet.)

Advertisements

Posted in Electronics, Hawaii, Kite Aerial Photography, Photography, Photosynth | Tagged: , , , , , , | Leave a Comment »

Photosynth from a Kite

Posted by Tom Benedict on 12/06/2009

Photosynth is a web-based program that lets you assemble a number of pictures of a single subject and combine them into a 3D navigable scene built from your pictures.  It doesn’t create a 3D model, but it does arrange all the still pictures so that a viewer can move around the scene and see each picture from the angle it was taken at.  People have made some truly phenomenal synths of a range of subjects.  It’s well worth a visit.

Of course I had to try it from a kite.  I pulled out the raw pictures from a couple of previous KAP outings and came up with some decent synths:

As it turns out KAP is very well-suited to making synths.  To make a well-connected synth requires a number of pictures, capturing the subject from multiple angles.  By its very nature KAP takes pictures from a variety of angles and positions.  I’ve made a number of KAP flights specifically to generate a synth, but for the most part they have been the serendipitous outcome of a flight otherwise geared toward still photography.  These are some of the better ones, taken with Photosynth in mind:

But there are times when things just plain don’t work.  Photosynth doesn’t like water because picture-to-picture, water varies, so there’s nothing really to tie one image to the next.  I’ve had some oddball results from synths that included large bodies of water.  Special techniques need to be used.

One in particular, a synth from a set of flights over the Kiholo Bay Inlet, had a number of problems with it.  Discussion with the Photosynth development team led to the conclusion that it was the water, the lack of a good pool of overlapping images, and just a poor sampling of the area in general that led to the problems.  I’m planning a second trip to Kiholo Bay to try to take a better set of pictures for making a synth there.

An added side-benefit of Photosynth is that in the process of tying all the images together, it creates a point cloud indicating which points in the images tie to points in other images.  A good set of high detail pictures can generate a very dense point cloud.  It’s possible to extract the point cloud and use it to create a rendered 3D image of the scene.

Photosynth Surface Extraction

This model was generated using the point cloud from the Waikoloa Archaeological Site in the list above.  Considering there was no GIS data, no ground control grid, no real spatial or metric information of any kind, it’s remarkably accurate.  Though it also shows some of the problems with this method.  The road surface in the cut is quite smooth, so there are patches without any points in the point cloud.  These show up as gaps in the surface.  There are also some very rough areas of terrain on the same size scale as the mesh spacing in the surface.  This caused issues as well.  All in all, though, it’s a neat technique.

But the flight at the Kiholo Bay Inlet has been the most instructive of all.  Because of the sparse data set, it really pushed the Photosynth algorithm, and indicated some new approaches that would help with making synths from a kite.  When I go back, the plan is to:

  • Get the camera airborne over the water, and walk the length of the pond while pointing it at the far shore.  Take pictures every five seconds to get good overlap between frames.
  • Turn the camera 180 degrees and walk back the other way, taking pictures of the near shore.  Again, take pictures every five seconds to get good overlap between frames.
  • Turn the camera 90 degrees to face down the length of the pond, and tilt it down until the horizon is just out of view.  Walk the length of the pond taking pictures every five seconds.
  • Turn the camera 180 degrees to face back the other way down the length of the pond, and walk back taking pictures every five seconds.

At this point there should be enough images with enough overlap to make very good references for both shores, and enough images to tie the two shores together into a single frame of reference.  From here on out images can be taken to place them inside this frame of reference:

  • Get the camera to a good high altitude over the far shore, 50-100m or so, and begin taking hemispherical panorama sets.  Start at one end of the pond and work toward the far end, moving maybe 10-20m between sets.  This should provide a view in every direction from any point above the pond, and the high altitude should let the straight-down ortho shots tie into the frame of reference we generated above.
  • Make a second pass at a lower altitude with the camera over the near shore.  Similar spacing between hemispherical sets.
  • Finally, switch back to manual control and take detail sets of some of the features at the site (resting turtles, the turtle observation station, various rock features, key features along the shoreline, etc.)

At the end of the day, the pictures need to be culled to remove:

  • Blurries – Photosynth doesn’t work well with them, and they’re no fun to look at.
  • Bad exposures – For the same reasons as above.
  • Frames with more than 50% water in them – This was one of the key issues with my first attempt at this site.  Photosynth doesn’t deal well with water.

Finally, put all the images into Photosynth and see what comes out!

It’s a long process, and the number of images involved can be quite large.  Each set of images for the hemispehrical panoramas is 40 to 48 frames.  Ten spots along the pond can generate almost 500 images.  The two passes will crank that number up to about a thousand.  Throw in the initial framework images, and the total will probably exceed 1500 images.  This is more than I did at Green Sand Beach, by far my largest synth to date, but it should make for a very complete synth of the Kiholo Bay Inlet.

I should have a chance to try this over the weekend, if the weather holds.  Time will tell.

Tom

Posted in Hawaii, Kite Aerial Photography, Photography, Photosynth | Tagged: , , , , , , , , | Leave a Comment »