[clug] Stereo photography (was: Photo management?)

[Ian Munsie]

Hi Mike,

Stereo photography can become a rather tricky subject due to some
(unsolvable) technical issues I'll touch on a little below, but it can be
quite fun nevertheless.


Camera
------

The camera I mostly use for this is a Fujifilm FinePix Real 3D W3:

https://en.wikipedia.org/wiki/Fujifilm_FinePix_Real_3D

It includes two lenses separated by some distance similar to human eyes
(it's actually a little wider than my eyes) and takes two photos of the
same subject simultaneously from different perspectives (it has other 3D
modes as well, but nothing that couldn't be done with a regular camera). It
also has a glasses-free 3D display built into the camera ("sweet spot"
based, meaning you have to look at it straight on), which allows you to see
in advance how the 3D photos will look, and is handy to show subjects
themselves in 3D, which they always like.

It is also possible to take stereo photographs with any camera by taking
two photos from slightly different perspectives, but this can be difficult
to get the orientation right between the two, and if the subject moves (or
wind blows a leaf, etc) it means the photos will not quite match up between
both eyes.  There are various rigs available to remove some of the error
from this process.

It is also possible to use two individual (preferably identical) cameras
simultaneously if their settings (focal point, focal length, f-stop) are
identical and their shutters are synchronised. At some point, I'd really
like to try this set up using two DSLRs with "tilt-shift" lenses rather
than ordinary lenses as my experience working with stereo projections in
computer graphics leads me to believe that could result in a superior
stereo photograph if setup correctly with a known display size, but trying
that would be somewhat expensive and I have never heard of anyone else
doing it.



Viewing Options
---------------

There are a number of options available to view a stereo photograph, each
with their advantages and disadvantages: computer monitors, TVs or
projectors using either active shutter glasses or passive polarised
glasses, anaglyph (red-cyan) glasses with any display, displays /
photographs with a lenticular lens array over the top for glasses-free 3D
viewing, or just simply using the cross-eyed or distance viewing techniques
to see a 3D photo with no special display, or by using the mirror technique.

I personally have a laptop with a 3D display (no longer being
manufactured), and a 3D DLP projector (BenQ W1070).

3D computer monitors usually use nvidia 3D Vision and are 120Hz (or higher)
active displays and the V2 ones feature a low-persistence backlight (turns
off while the glasses change eyes to reduce crosstalk and increase the
perceived brightness). These use nvidia's proprietary active shutter
glasses, which are 60Hz per eye. These types of displays are a pretty good
choice, but do suffer from some degree of crosstalk, and depend on nvidia's
proprietary drivers (also, for Linux I believe that a Quadro card may be
required from the documentation, though I have seen reports that it might
be possible to make it work with a GeForce card like we do in Windows).

3D televisions have several different 3D formats they may use. side-by-side
is usually the easiest option (though not necessarily the best as it halves
the horizontal resolution) and is supported by geeqie and mplayer. 3D
televisions are a poor choice for stereo content as they tend to suffer
from exceptionally bad crosstalk thanks to the long time it takes the
pixels to change (that is, each eye can see part of the image intended for
the other eye), and they tend to have pretty high latency (fine for photos,
not good for gaming), but have the advantage that they are fairly common
and you may already have one. Which glasses they use and whether they are
active or passive will depend on the specific TV. I believe that some use
DLP glasses, which are standard.

For 3D projectors we only really consider 3D DLP projectors. These are
similar to 3D TVs, but they are generally a much better choice - they have
zero crosstalk thanks to the speed at which the DPL mirrors are able to
switch (much faster than even the best LCD) and when used for gaming are
generally much lower latency than TVs. Their disadvantages are the space
required (short throw versions are available for smaller rooms), need to
keep the room dark (or use a rather expensive black projector screen), and
replace the bulb every now and then. The active DPL glasses they use are a
standard so you are not forced to use the projector's brand glasses, though
beware that the projector probably won't come with any and they will need
to be purchased separately. The IR signal used to synchronise the glasses
is emitted from the projector and simply bounced off the projector screen.

Given the typical screen size of a projector, these have the highest risk
of violating infinity for pre-rendered content (displaying an object
further apart than your eyes) and photos may require a parallax adjustment
to offset their left and right images to be able to comfortably view.
Movies already are calibrated for a larger screen (IMAX), so no need to
worry there (but 3D movies also generally suck as a result of this), and
games can calibrate to whatever screen size they are being used with for
the best result.

Anaglyph glasses are a low-cost option ($2 from ebay) that can be used with
any display, but I would not recommend this for anything other than trying
out 3D since the false colours and high crosstalk result in eye-strain. I
cannot tolerate anaglyph for more than a few minutes, whereas I can
comfortably wear active shutter glasses all day with 3D games. geeqie and
mplayer can both output stereo content in several forms of anaglyph
(compromising between more realistic colours and less crosstalk between the
eyes).

Displays with a lenticular lens array do not require glasses to view - the
Fujifilm camera I use has one of these on the back. They usually require
the viewer to have their head in a specific position ("sweet spot")
however, though there are some that use eye-tracking to compensate for this
in real time and can support a very small number of viewers anywhere in the
room (I'm not sure if any of those are consumer grade yet though).

Fujifilm also produces a 3D photo frame that is aimed at users of their
camera with the same sort of lenticular lens array over it. I have yet to
purchase this as I have my doubts as to it's general usefulness since the
fact that it still has a sweet spot means the viewer must stand in a
specific spot and cannot enjoy the photos from anywhere in the room.

It is also possible to print out a photo with the left and right views
interlaced and place a lenticular lens array on the photo itself, allowing
for 3D prints. Fujifilm has a service to do this, but it is not available
in Australia and I have yet to track down an alternative print service
available here. Apparently it is possible to purchase the supplies to do
this yourself.

The cross-eyed and distance viewing methods do not require any special
displays as they are simply a technique you can use to view a pair of
stereo images placed side-by-side. The images must be fairly close together
and should not be more than about 7cm or so wide, perhaps even less. The
further apart the images are on the screen, the harder these techniques are
to achieve. These will not give you the full impact as using glasses with a
full 3D display, but they don't cost anything and with a bit of practice
can become easy.

This is an example of a photo I took with the left and right reversed for
cross-eyed viewing. The trick is to go cross-eyed until the two images
merge into one. To help practice this technique, hold your finger up half
way between your face and the display and look at your finger instead of
the display. Focus on your finger and slowly move it forwards or backwards
until the images on the display behind it have merged together, then try to
refocus your eyes on the 3D image without pointing them back at the screen.
It may take a few attempts while you get used to the technique.

http://valen.darkstarsword.net/photos/stereo/workshops/crosseyed/reaching.html

This image is set up for the distance viewing method. For this method you
need to relax your eyes and allow them to defocus from the screen and look
behind the display until the images merge, then try to refocus on the image
without looking back at the screen.

http://valen.darkstarsword.net/photos/stereo/misc/distance/DSCF0292.html

The mirror technique works by placing a mirror in front of your nose (in
this case facing to the left) so you can see a reflection of the image in
the mirror. Focus on the image in the mirror and it should pop into 3D.
This can be easier than the above techniques since it does not require your
eyes to be looking in a different direction to their focus, and can
comfortably be used to view larger stereo images, though it can be
difficult to fit the entire image in the mirror (you may have to move your
head back or forwards). Also, since most mirrors are imperfect (especially
at this angle) they may show a double image:

http://valen.darkstarsword.net/photos/stereo/misc/mirrorl/DSCF1270.html



Subjects
--------

I've found that there are certain subjects that work well in stereo that
don't work at all in 2D, yet just as many that work better in 2D than 3D.
If you ever see a scene that looks really interesting to your eyes, but
plain and uninteresting on a 2D photo when the depth has been lost (or
replaced with a depth of field blur) it might just be a candidate to try in
stereo - here's a good example of this:

http://valen.darkstarsword.net/photos/stereo/misc/crosseyed/DSCF0074.html

In 2D all the rocks blend together and it becomes a plan and uninteresting
shot, but in 3D the individual rocky outcroppings can easily be
distinguished from one another and the shot is interesting. Here's another
example:

http://valen.darkstarsword.net/photos/stereo/misc/crosseyed/DSCF1991.html

In 2D there is nothing interesting about this shot and I would delete it,
but in 3D the depth of the hole is apparent and the shot is interesting
(still not really a keeper, just interesting to show the 3D).

If the subject will not gain much from 3D, it may be better shot with the
additional control that a DSLR provides in 2D and without the technical
problems that stereo photography brings. 3D tends to work better for closer
subjects rather than those further away, and when the subject links
multiple depths together.

If the subjects are too far away or too far apart they may appear as
layered 2D images, which can be ok, but does not really do stereo
photography justice.  Zooming in on a distant subject with the camera will
not provide the same stereo effect as moving closer to it (the same thing
happens in 2D - you might be familiar with the dolly zoom effect, but in 3D
it is far more pronounced).

For instance, this photo did not gain much from being shot in stereo as
everything is just too far away and the effect is not very pronounced
(displaying this on a larger screen may help a little):

http://valen.darkstarsword.net/photos/stereo/misc/crosseyed/DSCF1318.html

Stereo photography can work especially well to show detail that is lost in
a 2D image - most photographers will see running water and immediately set
their camera to use a longer exposure time to get that classic artistic
streaking effect, but in 3D you might do the opposite and try to freeze the
water in the frame so you can examine it's structure in detail (I have
better examples, but not that I can post here):

http://valen.darkstarsword.net/photos/stereo/misc/crosseyed/DSCF1317.html

In video games, playing in stereo brings out a lot of detail that players
would usually ignore - grass, leaves and rocks are no longer just there to
"not look weird because they are missing" - they now have real detail and
players will stop and admire just how much effort the 3D artist put into
them (or in some cases how little). The same works in a stereo photo - if I
were taking these in 2D I would probably have focused on an individual
flower or leaf and used depth of field to emphasise it, but in 3D the wider
scene is interesting as the detail on every single flower, leaf and blade
of grass is apparent (if possible, best viewed on a larger screen to see
the detail more clearly):

http://valen.darkstarsword.net/photos/stereo/misc/crosseyed/DSCF1217.html

http://valen.darkstarsword.net/photos/stereo/misc/crosseyed/DSCF1224.html

http://valen.darkstarsword.net/photos/stereo/misc/crosseyed/DSCF1214.html


Issues
------

The Fujifilm camera should only be used in landscape orientation when both
lenses are used since the lenses must be aligned horizontally - otherwise
the images will be misaligned between the eyes and will cause eye-strain
and will not be pleasant to view in stereo (if possible at all).  This can
be corrected in post to some extent, but only to a point - if the photo was
a full 90 degrees out it will not be possible to correct (you could still
salvage either of the two images as a 2D photo).

That's not to say that portraits can't be taken in stereo, but the lenses
have to be aligned horizontally, whether that means using a different rig,
or taking a wider angle landscape shot and cropping it to portrait.



A stereo camera sees the world in much the same way our eyes do:

\   \    /   /
 \   \  /   /
  \   \/   /
   \  /\  /
    \/  \/

But the problem is that is not beamed directly into our eyes, but rather
has to be displayed on an intermediate display and we don't quite see that
display the same way. There's not much that can be done about this in
photography or fimography, which is one of several reasons that 3D movies
are usually not considered to be very good. I do think that a pair of
tilt-shift lenses could help here, but even that would not help with the
fact that we do not know ahead of time what size display will be used to
view the image later.

The reason the display size is important, is that if the left and right
images of an object is displayed on the screen further apart than the
viewer's eyes (regardless of how far away the display is), the object will
appear to be beyond infinity, which quickly becomes uncomfortable or
impossible to view. The only way to combat this is to shift the offset of
the two images until nothing is more than 7cm apart on the largest display
it might ever be displayed on.  Displaying the content on a smaller screen
will quickly diminish the strength of the stereo effect - therefore, the
IMAX theatre in Sydney is another reason that 3D movies are considered poor
as their 3D effect is reduced on anything other than the IMAX theatre in
Sydney, and by the time you are viewing it in a home theatre there is
almost no 3D left.

But video games do not suffer this same problem - they are rendered live
and know the size of the display they are being rendered on, and can use
this information to skew the projection so the viewing frustrum for each
eye will touch the edge of the screen at the point of convergence, plus
they can dial the overall strength of the 3D effect and the point of
convergence up and down as desired:


 \-    \                           /    -/
   \-   \                         /   -/
     \-  \                       /  -/
       \- \      screen of      / -/
         \-\     known size    /-/
            \-----------------/   <-- point of convergence
             \\-           -//
              \ \-       -/ /
               \  \-   -/  /
                \   \-/   /
                 \ -/ \- /
                  o     o

3D screenshots of games are still a problem however - if they are scaled up
to a larger display they may violate infinity, and if they are scaled down
to a smaller display they will have a reduced 3D effect. Now that you know
this, here are some screenshots I have taken in various games that is
calibrated to a 17" display for a comparison of how they look compared to
the photos:

http://photos.3dvisionlive.com/DarkStarSword/

Without the nvidia plugin that site is pretty useless, but I made a user
script to add a download button to it to get at the raw side-by-side images
that can be saved as .jps files and opened with geeqie:

https://github.com/DarkStarSword/3d-fixes/raw/master/3dvisionlive_download_button.user.js


Cheers,
-Ian


-- 
http://darkstarsword.net
http://darkstarshout.blogspot.com
http://github.com/DarkStarSword
--
Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html