Category Archives: Stacking

Philosophy of Advanced Stacker PLUS

ASP14e-500

Coming soon…

Our 14e release is imminent. But after looking at our survey results and the most recently reported issues, we thought it wise to provide some hints to help you use the current software and to help you understand tradeoffs we made.

Our most commonly reported issue is something we struggled with. In an attempt to help users understand features we caused confusion. What are we talking about?  There are pop up windows: some of which should be Continued some of which should be Stopped. The only notification tool that works across all versions of Photoshop reliably doesn’t allow much customization, so when confronted with each of these dialogs, its understandable why some people click Stop when they should Continue and click Continue when they should Stop. The pop-up window below was the worst. We added highlighting to make it easy to see the difference, but we can’t use color or graphics on the actual pop-up without creating compatibility problems.  Do you notice the difference?

ASP14d_NotStop

This dialog indicates that the Watermark Alignment is not set… click CONTINUE

ASP14d_DoStop

If you stop on the first dialog, the layers needed to do stacking are not created and you will get errors the first of which is usually “Move is not currently available”

Move Not Available

Move Not Available

Once there is any kind of error it’s time to Stop and restart because manipulating layers in Photoshop is a little fragile.  The good news is the first pop-up can be turned off. The better news is that in 14e we have already turned it off for you. In fact in 14e we’ve trimmed down interaction as much as possible but still kept the awesome extra features.  Here is how to turn off the first pop-up (the one regarding Watermark Alignment) in version 14d and earlier.

Stop the Pop

Stop the Pop – disable the “Stop” step in Align My Watermark

Power of the Batch

The Advanced Stacker PLUS derives much of it’s power from two key things: Photoshop’s ability to batch process images AND Photoshop’s ability to open just about any image format on the planet.  Batch processing works because it is possible to use File -> Automate -> Batch to hurl a handful or a folder full of images at customized scripts and actions.  While Adobe Photoshop Elements has a “batch processor” there is no way to do operations other than those that are built-in to the software and that’s why ASP doesn’t support Elements.

One of the weaknesses of using Photoshop’s captive power is that it requires exploiting the tool in ways that are “allowed”.  While we like Adobe Photoshop CC, there are still many people who are very happy with older, non-cloud versions of Photoshop like CS3 or CS4.  We chose to not abandon that 35% of our customers so we support ALL CS versions of Photoshop on both Mac and PC. and that means we limit our actions and scripts to features that are common across all those versions and platforms. Readers may not be aware, for example, that while Adobe has published tools for customizing the interface, those tools have generally only supported the “latest” version of software. Indeed one of the most used tools (Configurator) has been abandoned and is no longer supported.

Another tradeoff has been in how we document our features. For example the File -> Automate -> Batch method of stacking a folder full of files is workable, if inelegant. But we use the Photoshop Bridge method of feeding the stacker and like it much more for several significant reasons.  One huge advantage of using Bridge is that you can actually see the content of the files. Another obvious advantage is that you are not constrained by what’s in your folder. If you have 29 shots from one sequence and 120 of another sequence you don’t have to split those shots between folders. And, in fact, with Bridge to select files you don’t have to use the same stacking method for all the files of one set, or have files from only one folder!

A word about 14E

As noted above, version 14E is imminent. Surprisingly the biggest obstacle has not been the additional features, the primary obstacle has been packaging and delivering the content. Windows 8.1 and Mac OSX 10.9 have gotten very protective of their machines and throw up many roadblocks to try to keep your machine safe from viruses and trojans. This means we had to invest $1500 dollars to become an LLC, get signed up for the Mac developer plan, get a Mac code signing certificate and get a code signing certificate for Windows, too! That doesn’t include acquiring a Mac or polishing the scripts and installers on each machine. Here is what the problem looks like on a Windows PC using Internet Explorer without signed code (it’s just as bad with Safari on Mac).

ASP_IE_RunAnyway

In fact, on both machines even though the code is signed (proving its provenance), you are still likely to get a warning like “This is not a commonly downloaded file”. It might be easier if we could email it to you, right? Except that Google, Yahoo and many others will not deliver an email that includes any executable content.

Both PCs and Macs have reached this point described in a Mac advertisement from 2009 that pokes fun at Vista’s intrusive safety system. Guess what… both machines are becoming like this because there are so many, uh, jerks out there eager to harm you electronically.

Our holy grail has been to create a single deliverable package that works both on a PC and on your Mac that we can document clearly, simply and as completely as possible.  That was probably too high a goal.

What’s coming in 14e?

  • Installation now is as simple as clicking.
  • EXIF data for the first image is preserved
  • New stacking mode of Ultra Streaks
  • Streamlined pop-ups to the minimum
  • Fully supports paths on both PC and Mac
  • Installation works for ALL versions of Photoshop CS you have installed on your machine
  • A price increase. But current owners will get the upgrade for free.
  • Some features we’re keeping under wraps for now.

Trimming away the Excess (Photoshop)

I created a problem for myself twice and with the Total Lunar Eclipse coming in April, 2014 I suspect I’ll be creating the same problem again.  I wanted to record a time-lapse of the May 20, 2012 Annular Solar Eclipse as well as the June 5, 2012 Transit of Venus.  I used a solar filter and an Equatorial Mount to help me track the sun. Unfortunately getting a good polar alignment during daylight is beyond my skill set. Without good alignment I had to manually repoint the telescope rather frequently. It is not necessary to understand any of the gobbledygook you just read except to know that the sun was MOVING from frame to frame – see the image below.  With all that movement, a time-lapse looks like a jitterbug dance.  In fact, this artsy composite shows just how much the sun moved around in my frame – that is, all over!

Solar Art

Annular Solar Eclipse with un-centered and trimmed frames. AKA Solar Art the “dots” on the sun are large sunspots.

My friends suffered from similar problems and each of them undertook automated solutions to the problem using Photoshop or something similar.  My goal was to solve the problem in a generic way and along the way I learned some useful additional Photoshop tricks.

The Solution

I used the trim feature of Photoshop.  You can find Trim under Image -> Trim.

Trim makes note of the current value of the upper left (or lower right) pixel. It then creates a selection that includes all of the rows and columns that have the same value and inverts the selection. Finally it crops off all of the selected area into the smallest possible rectangle.

An easy way to think of this is: imagine a dark photo with a white frame around it.  By using “trim” the entire white frame will be cut away.  The same approach works if the frame border is black, blue, transparent, and so on.

But there is a catch!  There is no tolerance setting for the trim value so whatever needs to be trimmed must be an exact color match. Unfortunately this presents a problem because even in a photo of a black sky, the black areas are not uniformly black.  Some values may be 0,0,0 but others 1,2,1.  And then there is noise!  Even though you may not notice a difference between two adjacent pixels trim only operates on those pixels that have EXACTLY the same value as the upper left or lower right.

To get trim to work in a reasonable fashion, therefore, we must turn all of the almost black pixels into black (or some other color).  I feel a Photoshop trick coming on here. Our trick is to use a duplicate layer and adjust it to cause it to trim the way we want. We will discard the duplicated layer when we are done.

Trim-o-Matic

  1. Open the image.
  2. Duplicate the image as a new layer.
  3. If the duplicate layer is a smart object, convert it to a raster layer. (To convert a smart object to a Rasterized Layer, open the layer palette, right-click on the duplicate layer and select “Rasterize Layer”)
  4. On the duplicate layer use
    “Filter -> Noise -> Dust and Scratches” with Radius = 3 and Threshold = 3.
  5. Make sure your foreground color is black (x is the hot key).
  6. Use the “Fill” tool (paint bucket) with tolerance set to 40, Opacity 100%, Mode = Normal and no options checked.  Click the extreme upper left pixel. Fill will replace all of the outer area with black. You may want to click multiple times.
  7. If you have some areas that are brighter than the object, you may also want to apply a brightness and contrast adjustment.
  8. Use the trim function on the current layer with all boxes checked and “Top Left Pixel Color” selected.
  9. Trimming changes both the working layer and the original layer below it.
  10. Discard the working layer.

Here are the steps in illustrations.

Rasterizing a smart object

Rasterizing a smart object

PS_DustAndScratches

Dust and Scratches settings

gs_2014-03-23_084744

Filling with black

Filling with black (might need to repeat this)

Dust and Scratches + Fill may not be enough. Fortunately the duplicate layer can be severely adjusted if needed since it will be discarded.

 

Trimming Tool

Trimming Tool

Select Areas for Trimming

Select Areas for Trimming

 

After Trimming

After Trimming

Delete the Extra Layer

Delete the layer that was created for the sole purpose of trimming

 

What If It Crops Too Tightly or Inconsistently?

This set of steps written as an action can be used to automate the process of trimming. There are still a few remaining problems to work out. Sometimes after trimming the total horizontal or vertical pixels of the object the resulting image size will differ by one or two pixels. To solve the “dimension” problem, the simplest method is to use Image -> Canvas Size. Set the canvas to about 20 pixels larger in each direction. Select “center” (the dot in the middle next to anchor) and set the fill color to match the background.

Expand the canvas and center the image

Expand the canvas and center the trimmed image

After Trim and Canvas Size

After Trim and Canvas Size

The above procedure will work quite well for animating a sequence of sun or moon shots – except for eclipses.  Eclipses don’t work properly because an edge of the sun or moon disappears leaving no edge to properly orient with the others.

What can we do when we no longer have constant edges to align with?  Divide and conquer! Instead of trying to apply the same action to all of the images, we will can change edge we select when enlarging the canvas size. We can orient sets of images based on which part of the image remains constant.   Which edge or corner should you pick? Pick the edge that stays the same (if there is one!)  For an Annular solar eclipse, at least one of the directions will never be darkened – that’s the one to pick!  The annular eclipse sequence shows that the upper left limb and lower right limbs of the sun remain present in all of the shots.

Field Rotation

There may still be another problem: Field Rotation. If you’re thinking that perhaps this has something to do with improving crop yields on a farm, sorry to disappoint you. The Annular Solar Eclipse and the Transit of Venus were events that took place over a period of from 3 to 5 hours.  During that period the earths rotation causes the sun, moon and stars to move. It also causes them to “turn” as viewed from terra firma.  You can see this for yourself if you watch the full moon from moonrise to moonset. At moonrise make note of the orientation of the “man on the moon” and compare it with the orientation at moonset. Go ahead and watch. I’ll wait for you.

So what do you do if you have this field rotation?  Either live with it and accept that the animation won’t be entirely accurate, or you’ll have to do a much more complicated set of operations by progressively rotating the images.  That’s more than we want to tackle, so you’re on your own for that!

The Results

The Annular solar eclipse sequence. The Abstract Solar Art image at the top of this article was created from the un-centered and trimmed frames. Obviously this wasn’t perfect – in part due to clouds and shimmer in the atmosphere.

Flying Dust Monsters (and Venus, too)

Not seeing the above video? Click here to watch.

Obviously this one had some trimming errors… and nasty dust on the sensor but this is the once in a lifetime Transit of Venus.

The Revenge of Lens Correction

There are plenty of ways to make your images look weird.  Some of the perturbations are due to sneaky little things that Adobe Photoshop, Adobe Lightroom and/or Adobe Camera RAW might be doing to the data.  We already talked about the “cooking” that is applied by default to RAW images and why letting that cooking stand unchallenged may be a bad thing. We’ve even warned you about Blur and Jaggies that may NOT in fact be in your images.

Recently Dan Barr asked us what we thought was causing a problem in his stacked star trails. If you read the title you’ve probably already figured out the culprit… Lens Correction!  You may not notice anything weird if you process only a single image, but what if Star Trails, or image stacking are what floats your boat?

Notice the strange pattern in the upper left. This image is cropped from a larger image.

Notice the strange pattern in the upper left. This image is cropped from a larger image. Image by Timbo2013

Look in the upper left of the image above. That cross hatching is one possible artifact.

Why does this happen? The lens correction is a mathematical model that moves pixels around. Not surprisingly, since the images change – even if slightly, the results vary slightly, too.

How do you fix the problem?  Don’t mess with your images before you stack them.  Save the lens correction, contrast adjustments and other tweaks for after you’ve finished stacking.

Here is a before and after comparison:

danBarr_moire_marked

Notice the odd “Moire” like pattern above and to the right of the mountain? (Image courtesy of Dan Barr)

It’s a little subtle. Here is the weird part close up – notice the vertical undulations? The oddness somewhat resembles sensor banding noise except when you look at a larger scale, the lines are concentric.

ConcentricLines

 

By redoing the stacking operation without performing lens correction, Dan was able to get an image without the waves:

Stacked first, then adjusted – no moire!

With the strangeness vanquished, Dan was able to improve the brightness and contrast as well.

Tips for Using Layers for Stacking

Down Range

With hundreds of users of our Advanced Stacker PLUS, we get questions about workflow. What is the best way to manage files in Lightroom? What order should I perform operations, etc. While we have briefly given some hints in the comments for the Advanced Stacker, it seemed time to elaborate.

Of course it all begins with how you choose to organize your files. Organizational tips will come in a separate article as it would make this article too long.  Unfortunately Lightroom is not a particularly helpful tool to use for creatively stacking shots and that’s one of the reasons I don’t much like it. For my money Adobe Bridge is more powerful and flexible.

One particularly insightful exchange was with Dan B who wrote:

One minor gripe I have mostly has to do with incorporating the action into my existing workflow. For star trails my current workflow starts in Lightroom where I do any preliminary adjustments/corrections and then open the sequence as a layered document in Photoshop. I created a layer style which is my one-click way of changing the blending mode to lighten on all of the layers. Unless there is a way of applying the action to an already open layered document I would presumably have to adjust my work flow by moving the star trail sequence to its own folder, doing preliminary adjustments in ACR and then using the action as directed.

Beginning in Photoshop CS6, you can do a one button change of all blend modes and opacities of all selected layers. You can select all layers with Alt-Ctl-A  (Option-Command-A on a Mac).

PSMultiLayerOperation PSMultiLayerOpacity

The change applies to all selected layers.  Setting all the opacities the same won’t provide the nifty comet-style stacking.  There are some scripts out there to address variable opacity and other stacking tricks but there are restrictions. Installing a script isn’t always easy – depends what version of Photoshop you have, and no script I’ve seen is able to work from files. Scripts work on Layers. Scripts by design aren’t really meant to tackle repetitive tasks against a large number of files – that’s why actions were created.

Summary: There are two options: work on layers (actions or scripts) or work on files (actions).

There is no equivalent in Lightroom to do Photoshop Batch operations like you can from Adobe Bridge, and that’s unfortunate.

Another observation is that loading files into a stack in Photoshop and then editing individual layers is, SLUGGISH unless you have few layers – see later for just how sluggish.  Why would you edit layers? You might edit individual layers to remove things like airplane trails or bright light sources. Airplane trails are a lot easier to remove from individual documents than to remove after the stack has been created. However my recommendation is: don’t do editing in layers. Export the documents to TIFF or JPG and edit them individually. Why? Because then you can use the edited files standalone or stack them in different ways and NOT have a huge layered document. You also have more flexibility to use programs like StarStax which do not work well with RAW files (StarStax does not know how to apply all of your ACR or Lightroom edits). Alternatively, you can load files into a layered document, edit the layers and then use a script (Export Layers to files) to save layers as individual files.

Load Into Layers from Lightroom

Lightroom_OpenAsLayers

Save From Layers into Individual Files

PSSaveLayersAsFiles

If you save from layers into files, Lightroom will not know where those saved files are unless you Import or Resynchronize the folder where the files are saved. Bridge, on the other hand, doesn’t require importing or synchronizing – though you may need to do a refresh operation. By the way the Photoshop “Export Layers” operation insists on adding a number prefix to each of your files… and it will make a mess if the opacity of each layer is not 100%.

Speedy?

If you want to do your stacking using layers, there is one more thing that might change your mind… speed.  We ran speed tests on two different machines comparing the end-to-end time needed to create stacked documents using the layering method with the automated Advanced Stacker Plus.  We were shocked by the difference.  For each machine we stacked 60 RAW files from a Canon 5D Mark II. In each case we applied a linear adjustment to all files, and made a tweak to the color balance.  We drove the layering method using Lightroom 5’s “Edit as Layers” operation. For the Advanced Stacker PLUS we drove the operation using Adobe Bridge CC. We kept Bridge CC, Lightroom 5, and Photoshop CC loaded in each machine so that the same starting memory footprint was used. And to make sure there was no advantage from using pre-loaded files, we used a different set of 60 files for each comparison.  The file sizes were identical.

The less speedy machine was a quad core AMD Phenom II processor with 6Gb of memory, Windows 7 Home Premium, 64 bit. The new machine was an Intel I7 quad core machine with Windows 8, 64bit, and 12 Gb of memory.

On the lower end machine it took 33 minutes to load 60 Raw files as layers, change all the layers to blend mode Lighten* and merge those layers. There was HEAVY swapping and the machine was extremely sluggish.  Using the Advanced Stacker PLUS to perform the same result took 8 minutes and the machine never became unresponsive – because there are never more than a dozen layers in memory.  The Advanced Stacker PLUS took 76% less time!

*NOTE: In Photoshop CC and Photoshop CS6 all the blend modes are changed with one Select All Layers command and one blend mode change.  On the sluggish machine it took almost 2 minutes for the “Select All Layers” key sequence to complete!

On the faster machine the results were similar: it took 19 minutes to load all the layers, change the blend mode and merge the visible layers into a single image for saving while it took 3 minutes to use the Advanced Stacker Plus.  On the 12 GB machine there was some pretty heavy disk operation going on when using layering, but memory did not top out. The Advanced Stacker PLUS took 80% less time.

In each case, adding more layers will make the stacker speed advantage even greater because once the machine maxes out memory it becomes a performance dog.  We’d love it if you’d run a comparison on your hardware to see what your results are like.

In Summary

There are many folks out there who are proponents of stacking star trail shots using layers. I’m not a fan.  Certainly using layer provides some benefits, but it also comes with some (high) costs.  Here are some tradeoffs to help you decide whether layering shots in Photoshop will be more effective for you or not:

Layer When

  • You have LOTS of memory and patience.
  • You have fewer than about 30 layers (shots) or your individual shots are small.
  • You don’t expect to save the final image as a layered document (only .psb files allow sizes big enough to hold a typical layered document)
  • You don’t mind throwing away any editing you do in a single layer (e.g. removing plane trails, stray light, etc.)
  • You don’t mind manually updating blend modes and opacity or finding installing and using the (very few) tools available to help with layer adjustments.
  • You intend to do something totally different from all the currently popular effects (Comets, streaks, etc.)
  • You don’t plan to make a timelapse – or if you do, you can live with the restrictions created by a layered document.

Stack via an Action or External Tool When

  • You have a boatload of images or limited memory.
  • Want to create intermediate images for timelapse/animation.
  • You intend to pre-edit individual frames to clean them up before creating a final version.