The View Up Here

Random scribblings about kites, photography, machining, and anything else

  • Flickr Gallery

Archive for the ‘Photosynth’ Category

More Fun with Panoramas

Posted by Tom Benedict on 27/04/2010

I got out a couple of times over the weekend to shoot more panoramas.  One that I did over Puu Huluhulu on Saddle Road turned out quite well.

Kipuka Puu Huluhulu and Mauna LoaThis was done just after 7am, with a really steady breeze coming in from the Hilo side.  The panorama consists of 21 photos taken in two stripes, something only possible with steady wind and a good solid kite.  It’s something of a milestone for me for a number of reasons:

  • I’ve been trying to photograph Puu Huluhulu under these conditions for over a year, and for over a year I’ve been thwarted.
  • This was the first time I’ve been able to really test Autopano Pro since I picked up the license last week.  I’m overjoyed by how well it worked.
  • This is the last time I ever want to depend on the wind to be just right to let me make a two-stripe panorama.

That last bit probably takes some explanation.

I long ago quit making kite aerial panoramas the traditional way, in favor of a method developed and proven by a French KAPer who goes by the name of Vertigo in the KAP forums.  His method, which he calls “Burst autoKAP” involves spinning the rig at a fixed rate and holding the shutter button down so the camera can take pictures as fast as it can store them.  With a good high shutter speed, this method works flawlessly.  Using the traditional method, a 360 degree panorama took roughly a minute.  That’s a whole minute during which the wind can change, the camera can drift, and things can get out of alignment.  Using Vertigo’s burst method, a 360 degree panorama takes my rig and camera roughly ten seconds.  The resulting panoramas stitch much more easily, with fewer errors and fewer panoramas lost to camera movement.

But your panorama can only be as tall as the vertical axis of your field of view.  Unless, of course, you shoot two stripes of photos at two different camera tilts.  A two-stripe 360 degree panorama takes my camera and rig twice as long as a single stripe, or twenty seconds.  While far better than the traditional approach (which would take upwards of two minutes), it’s still longer than the ten seconds a single stripe panorama can be done in.  Twice as long, in fact.  With twice the chance that the wind will shift, things will move, and the panorama will be spoiled.

Later in the weekend I made an attempt at a panorama of Ali`i Drive in Kailua-Kona.  The light was wrong, and the wind was gusty and light, but it let me test the concept and scout a better location to fly from when the conditions warranted it.  But the wind was too variable to allow me to try a panorama in two stripes.  Had I been relying on that technique for capturing that image under perfect conditions, the day would have been a dead loss.

So let’s go back up to that statement I made about the panorama only being as tall as your camera’s vertical field of view.  Panorama photographers have known this since they started assembling panoramas off of photographic print.  The solution is simple: rotate your camera 90 degrees so your wide axis is vertical, and you get more coverage.  It results in a slightly slower burst pattern in order to maintain the same percentage of overlap between images, but it’s far better than the 2:1 time penalty you take making a two-stripe panorama.  So the next question is how to rotate the camera in the KAP rig.

There are two approaches.  One is to change out the tilt bracket for one that holds the camera vertically.  The other is to add in another axis of motion to the KAP rig that allows the operator to rotate the camera’s orientation on the fly.  I went with this second option.

There is no pre-built kit of parts for doing this.  Most people who build a rig with this horizontal/vertical orientation axis build them on their own.  There are a handful of examples out there, but no one yet makes a kit where you can just click “HoVer Option” and get the appropriate parts.  Even so, the parts that are available from Brooxes can get you very very close.  That’s where I started.

My approach is to use my existing tilt bracket for the camera mount.  I added a utility frame and a Brooxes Better Gear Guide with 1:1 gearing in it for the HoVer axis.  This then replaces the tilt frame in my existing KAP rig, and the whole thing is re-balanced.  That was the idea, anyway.  Unfortunately life rarely works out the way you plan.

For starters, the 1:1 gearing won’t bolt straight in.  There is a clearance issue between the bracket and the side of the servo.  Likewise with everything in the normal orientation, there’s interference between the side of the servo and the utility frame as well.  Milling the required clearance was a quick job on the mill, though an equally good job could be done using a hand nibbler and a file.  I don’t own a nibbler, and the one we keep at work was… well… at work.  It was quicker to mill the parts out.

Next, there is no standard part to go from a Brooxes Better Gear Guide to the chopped-up remainder of a tilt frame.  The BBGG includes a nice aluminum gear hub that gave me the idea for how I wanted to mount the remains of my tilt frame: a 0.875″ diameter piece of aluminum, threaded to take the BBGG shaft with a set screw to lock it in place, and a bolt hole pattern to attach it to the tilt frame.  This can be made quite thin, and should keep the camera from overhanging the end of the BBGG too much.

From this point on it’s a matter of clearance.  I had to un-do a number of modifications I’d made to my KAP rig in the sake of keeping things compact.  The clearance required for the HoVer axis is quite large.  But after careful measurement, I think I can squeeze everything in.

Sorry, no pictures yet.  It’s still a work in progress.  But things are progressing well.  The rig should be finished before this coming weekend, and I hope to get a chance to put it through its paces in the air before Saturday morning.  If things work out well, I hope to get out early again this coming Saturday and catch that morning light on a new subject to go with the panorama I made of Puu Huluhulu.  But this time it’ll be using a single stripe of images and a camera rotated to vertical at the flip of a switch.

Tom

Posted in Engineering, Kite Aerial Photography, Machining, Photosynth | Leave a Comment »

Photosynth from a Kite – Take II

Posted by Tom Benedict on 16/06/2009

Last Saturday I returned to Kiholo Bay to make another attempt at taking a set of pictures specifically for making a Photosynth of the area.  Unfortunately the wind was much lighter than I was expecting, so the first hour was spent almost entirely with my rig on the ground.  The 6′ rokkaku I use for the bulk of my light wind flying stayed up the whole time, but the launch of the camera itself was delayed.  The wind did pick up later, however, and I got several hours of flying time in before the sun and the fact that I’d missed lunch by over an hour and a half finally drove me away.

Because of the shorter time table, I didn’t get to do everything I’d planned.  The first four passes, in particular, had to go.  But in the end I think this worked well since I’m not convinced they were necessary, or would even make for a good synth in the end.

Instead, I used a prototype autoKAP controller to take almost all the pictures used in the synth.  The prototype is loosely based on some code I had written for the Atmel ATmega168 microcontroller a few years ago, but it’s being developed for a much better, much more fully-featured controller that’s currently in the works.  Unfortunately I’m not at liberty to speak further on that subject, but it’s moving along.  Soon… soon…

The idea behind it is very similar to the idea behind the Gigapan camera controller:  Orient the camera through a series of tilt and pan angles so that the resulting images have good overlap, and can be stitched together into a large panorama.  Only in the case of Photosynth the idea is to get good overlap between pictures so that the viewer can look in any orientation from that single vantage point.

But this is where the requirements for Photosynth and the requirements for a panorama diverge:  In order to make a good synth, it’s not enough to have good overlap with adjacent images.  Images of the same part of the subject also need to be taken from different vantage points in order to provide parallax.  It’s this that lets Photosynth generate its 3D representation of the scene, and find all the spatial relationships between the individual frames that go into the synth.

In the case of the Kiholo Bay Inlet I started at one end and flew the camera up to a good working height, about 200′ above the ground and almost on the other side of the pond.  Once there, the camera took a hemispherical set of images, three images high and about 12 images around.  As the camera panned around to its starting point, I walked about 50-100′ along the pond and stopped so that another hemispherical set of images could be taken.  This process was repeated until I reached the end of the pond.  About this time the camera’s chip filled up, so it was time to switch.

The second pass was made at a much lower altitude, closer to 75-100′ above the ground.  I also moved back as far as I could toward Kiholo Bay, putting the camera over the near shore.  A similar set of hemispherical images was taken at several locations along the near shore, and at the lower altitude.   And again, about the time I finished at the last position the camera’s chip filled up.

All in all I took over 1200 images this way.  After rejecting images with large percentages of water, the few blurry frames that happened when I was taking in or letting out line, or frames that couldn’t be tied directly to another frame, I was left with about 860 images.  These went into the Photosynth software.  The result can be viewed here:

Kiholo Bay Inlet – Kite Aerial Synth

One surprising outcome of this set of pictures is that it did a better job of covering the area than the set I took the previous week where I was in manual control of the camera and rig.  This is mostly due to the brute-force effectiveness that autoKAP has of taking so many many pictures.  Even as I was going through the pictures and culling out the ones that wouldn’t work well with Photosynth, I was taking notes on which photographs to revisit from the standpoint of traditional still photography.

Paddlers Heading Out

There was a lot more activity at the pond this weekend than last, so I had some opportunities to photograph different groups of people kayaking, paddling, swimming, or in my case, flying a kite.

Wainanali`i Pond Mouth

I also had my first really successful picture of the mouth of the inlet where it connects to the sea.  The clouds and sun cooperated this time, and I was able to get some pictures that did a good job of indicating the depth of the water, the width of the channel, and the reason why tidal flow really doesn’t cause a great deal of mixing in the water.

Honu Swimming I

But my favorite part, by far, was the honu.  And this is where the limitations of autoKAP make it less than ideal.  Because the camera is moving on its own, it’s difficult to do intentional wildlife photography with autoKAP.  I’ve done plenty of serendipitous wildlife photography this way, but without some ability to aim the camera in a particular direction, making pictures of particular subjects is difficult. The handful of honu pictures I made that day were all done with the rig and camera under manual control.

Most animals, sea and land alike, don’t spend a lot of time looking straight up.  Once the kite is in the air and the camera has been lifted to a good working height, for all intents and purposes it’s invisible to creatures on the ground.  In this case the camera was roughly 20′ off the ground, and because of the line angle I was about 40′ from the honu.  As far as it was concerend, I wasn’t there.  I’m often asked if I use some sort of a video feedback system to see what the camera sees.  I’ve been tempted, but so far I haven’t strictly needed it.  A KAPer can develop a sense of where the camera is and what it’s pointing at, even without video feedback.  In this case, once I knew the kite and rig was flying well and wouldn’t do anything unexpected, I positioned the camera above the honu and off to one side, and started taking pictures as it foraged on the rocks.

At some point I’ll return to Kiholo Bay just to take pictures of the honu.  But this trip was all about the synth.  Having it come out as well as it did really put a smile on my face.

Tom

(Note:  I have been calling the Kiholo Bay Inlet by the wrong name: Wainanalii Pond.  This is how it is commonly labeled on maps, hence my mistake.  But during a conversation with Bobby Camara, he pointed out that the village of Wainanalii is a two hour canoe ride to the north of Kiholo Bay, and that it was destroyed by the same lava flow that created the inlet.)

Posted in Electronics, Hawaii, Kite Aerial Photography, Photography, Photosynth | Tagged: , , , , , , | Leave a Comment »

Photosynth from a Kite

Posted by Tom Benedict on 12/06/2009

Photosynth is a web-based program that lets you assemble a number of pictures of a single subject and combine them into a 3D navigable scene built from your pictures.  It doesn’t create a 3D model, but it does arrange all the still pictures so that a viewer can move around the scene and see each picture from the angle it was taken at.  People have made some truly phenomenal synths of a range of subjects.  It’s well worth a visit.

Of course I had to try it from a kite.  I pulled out the raw pictures from a couple of previous KAP outings and came up with some decent synths:

As it turns out KAP is very well-suited to making synths.  To make a well-connected synth requires a number of pictures, capturing the subject from multiple angles.  By its very nature KAP takes pictures from a variety of angles and positions.  I’ve made a number of KAP flights specifically to generate a synth, but for the most part they have been the serendipitous outcome of a flight otherwise geared toward still photography.  These are some of the better ones, taken with Photosynth in mind:

But there are times when things just plain don’t work.  Photosynth doesn’t like water because picture-to-picture, water varies, so there’s nothing really to tie one image to the next.  I’ve had some oddball results from synths that included large bodies of water.  Special techniques need to be used.

One in particular, a synth from a set of flights over the Kiholo Bay Inlet, had a number of problems with it.  Discussion with the Photosynth development team led to the conclusion that it was the water, the lack of a good pool of overlapping images, and just a poor sampling of the area in general that led to the problems.  I’m planning a second trip to Kiholo Bay to try to take a better set of pictures for making a synth there.

An added side-benefit of Photosynth is that in the process of tying all the images together, it creates a point cloud indicating which points in the images tie to points in other images.  A good set of high detail pictures can generate a very dense point cloud.  It’s possible to extract the point cloud and use it to create a rendered 3D image of the scene.

Photosynth Surface Extraction

This model was generated using the point cloud from the Waikoloa Archaeological Site in the list above.  Considering there was no GIS data, no ground control grid, no real spatial or metric information of any kind, it’s remarkably accurate.  Though it also shows some of the problems with this method.  The road surface in the cut is quite smooth, so there are patches without any points in the point cloud.  These show up as gaps in the surface.  There are also some very rough areas of terrain on the same size scale as the mesh spacing in the surface.  This caused issues as well.  All in all, though, it’s a neat technique.

But the flight at the Kiholo Bay Inlet has been the most instructive of all.  Because of the sparse data set, it really pushed the Photosynth algorithm, and indicated some new approaches that would help with making synths from a kite.  When I go back, the plan is to:

  • Get the camera airborne over the water, and walk the length of the pond while pointing it at the far shore.  Take pictures every five seconds to get good overlap between frames.
  • Turn the camera 180 degrees and walk back the other way, taking pictures of the near shore.  Again, take pictures every five seconds to get good overlap between frames.
  • Turn the camera 90 degrees to face down the length of the pond, and tilt it down until the horizon is just out of view.  Walk the length of the pond taking pictures every five seconds.
  • Turn the camera 180 degrees to face back the other way down the length of the pond, and walk back taking pictures every five seconds.

At this point there should be enough images with enough overlap to make very good references for both shores, and enough images to tie the two shores together into a single frame of reference.  From here on out images can be taken to place them inside this frame of reference:

  • Get the camera to a good high altitude over the far shore, 50-100m or so, and begin taking hemispherical panorama sets.  Start at one end of the pond and work toward the far end, moving maybe 10-20m between sets.  This should provide a view in every direction from any point above the pond, and the high altitude should let the straight-down ortho shots tie into the frame of reference we generated above.
  • Make a second pass at a lower altitude with the camera over the near shore.  Similar spacing between hemispherical sets.
  • Finally, switch back to manual control and take detail sets of some of the features at the site (resting turtles, the turtle observation station, various rock features, key features along the shoreline, etc.)

At the end of the day, the pictures need to be culled to remove:

  • Blurries – Photosynth doesn’t work well with them, and they’re no fun to look at.
  • Bad exposures – For the same reasons as above.
  • Frames with more than 50% water in them – This was one of the key issues with my first attempt at this site.  Photosynth doesn’t deal well with water.

Finally, put all the images into Photosynth and see what comes out!

It’s a long process, and the number of images involved can be quite large.  Each set of images for the hemispehrical panoramas is 40 to 48 frames.  Ten spots along the pond can generate almost 500 images.  The two passes will crank that number up to about a thousand.  Throw in the initial framework images, and the total will probably exceed 1500 images.  This is more than I did at Green Sand Beach, by far my largest synth to date, but it should make for a very complete synth of the Kiholo Bay Inlet.

I should have a chance to try this over the weekend, if the weather holds.  Time will tell.

Tom

Posted in Hawaii, Kite Aerial Photography, Photography, Photosynth | Tagged: , , , , , , , , | Leave a Comment »