Generative AI and Photography
A Real-World Project
I've been writing about my adventures and experiments in Generative AI lately, with an expected bias towards Adobe Firefly. Adobe is where I work, after all, and I'm personally impressed on the way the company is approaching AI from the perspectives of transparency, responsibility and ethics. Adobe did not scrape the internet to train Firefly and as a result, the content generated by Firefly is designed to be safe for commercial use. That is the entire intention of building out this service, in mh opinion. If you can't use it in or for business purposes, other than sheer entertainment value, what's the point of bothering with generative AI?
Prior articles of mine have talked about practical uses for Firefly/Generative Fill and Generative Expand. But they've been more exploratory than true project application. The Art of the Possible, if you will.
Well, this week, I had the opportunity to do more than muse about the possibilities. This week, I needed Firefly to speed up my workflow and do things I likely couldn't have done, or done as well, as quickly.
And it all started with a "muckle-coo."
Introducing Willow, a Scottish Highland Long-horned cow, or "Muckle-coo," prior to any processing Lightroom or Photoshop.
On a recent trip to Prince Edward County in Eastern Ontario, we found our way to Topsy Farms on Amherst Island, near Kingston. Topsy Farms primary product is sheep's wool and products made from it. In fact, one of their blankets was featured in the hugely popular post-apocalyptic series, The Last of Us. We went in search of some famous dry stone walls and well, sheep. I wanted to add some sheep photos to my Stock photo collection.
We did find some walls, but instead of sheep, I found Cedar and Willow, Scottish Highland longhorn cows and official Topsy Farm ambassadors. For some time now I'd been wanting to photograph this animal and now I had two of them to pick from!
At the time of the capture below, I knew I did not like the background in the shot. It was far too busy and distracting. It would have to be dealt with. Other logistical issues would arise later. But Willow was pretty much perfect in terms of her pose. I waited patiently for Willow to move to a better spot in the field, but nope, she was happy to stay partly under the shed.
I did the bulk of my processing - as per usual - in Lightroom. A lot of that work was the removal of dozens of flies. A sad fact about cows is that they are incessantly tormented by flies. And those bugs don't make for a good close up photo.
The centre image shows just how many flies were on Willow.
After that work, I cropped for an 8x10, which is usually what I print to. Things were looking good! Well, except for the background. There are lots of elements to a working farm that are not necessarily photogenic. I was shooting with my OM5 - a micro four-thirds format camera - and even with a long lens and relatively shallow depth of field, the background was still too sharp for my taste. I was able to crop out some - but not all - of the distracting background elements.
I tried using the Remove Tool in Lightroom to clean things up, but with so much background to fix and very little clean reference material, I was not having much luck. I even tried the new Depth Blur tool in Lightroom Mobile (through the early access settings) and it did a pretty good job but wasn't as accurate as I would have liked when it came to determining subject vs background in certain areas. If I had just been pushing to social media - job done - but I was planning to print, and perhaps print large, so I needed the masking to be accurate all around. It was time to awaken the leviathan; Photo > Edit in...> Photoshop 2024.
In Photoshop, I started locally with the Remove Tool to get rid of the distracting metal reinforcements on the wooden posts. While the Remove Tool does use AI, it's done locally with no cloud processing and also uses the pixel content/resolution of the image to replace content. It does an excellent job more than 90% of the time.
Then it was time to set about simplifying the background. I tried the Remove Tool again, but much like Lightroom, there just wasn't enough original material to work with. So, I switched to the Marquee tool and set the the fixed size to a maximum area to 1000 pixels. Then it was just a matter of clicking on the image, positioning the Marquee and choosing Generative Fill from the new contextual taskbar. In a few minutes, the background was realistically cleaned up. I then did some minor touch-ups using the Remove Tool to rid the areas of the odd distracting artifact.
Removing a distracting background using Marquee selections Generative Fill.
Before and after of Willow. On the left is the image from Lightroom. Image on the right is after using Firefly within Photoshop to clean up the background.
Tip: Firefly's Generative Fill currently has a maximum resolution of 1024 pixels on the longest edge, so if your selection area is larger than 1024x1024, you could end up with some noticeable quality issues.
At this point I thought I was ready to go. Then I was informed that the customer wanted a 1-1 aspect ratio; square instead of rectangular. Even before the initial 8x10 crop, a square format would not have worked due to the length of horns. Backing up or zooming out when I was shooting the photo wasn't an option either as the fencing got in the way. It was time to add some height to the image using Generative Expand in Photoshop.
With the latest release of Photoshop, Generative Expand is available along with Generative Fill and out of the beta phase. Gen Fill is very easy to use; simply select the Crop tool to draw a new aspect ratio and choose Generative Expand from the Fill options. However, you're still limited by the 1024 pixel resolution of Firefly's Generative Fill.
Instead, I opted for the same technique I used with one of my landscape photos (discussed in this article), and similar to my other repair work mentioned above; multiple selections.
1) I converted the background layer into a standard layer by double-clicking on it in the Layers panel.
2) I used the Crop tool to add height (top and bottom) to the image.
3) I Chose the Marquee tool and set the marquee to a fixed size of 1000 pixels, with a slight feathering (5 or 6 pixels)
4) I clicked on the canvas to add a selection, making sure there was a bit of overlap with the original image.
5) I then Chose Generative Fill from the Contextual Task Bar, and clicked Generate, without any text prompt.
On the first try, Firefly and Photoshop gave me a variation I could use for each selection! The final result was realistic enough that - had I not shared this information with you, it's unlikely you would have known the work had been done.
Willow, ready for printing to a square canvas.
The Final Results
I took a similar approach with Willow's friend, Cedar, although in the case of that image, I did not need to extend the canvas to make a square format.
I am anxiously awaiting canvas prints of these two images to arrive.
Have You Herd...?
As luck would have it, a week or so later, I found myself with yet another opportunity to photograph Scottish Longhorns - and far more of them! Red Farm Lane is located off Highway 28, only about 25 minutes from the cottage. Along with the Highland cattle, on their 100-acre farm, Red Farm also raises and sells free-range turkeys and chickens, eggs, Black Angus cattle and pigs.
I had seen the longhorns in the past as we drove by to some other destination and I'm sure those sightings sparked my original interest. Of late though, they hadn't visible from the highway.
I thought it likely that the cattle had been moved to a different pasture to allow the other field to recover.
After my success with Will and Cedar on Amherst Island, I decided to visit Red Farm Lane and ask if I could take photos of their cattle.
Farm owners Terry and Lisa were kind enough to share their knowledge of Scottish Highland cows with me and Karen. Highland Cattle is Scotland's best known and oldest native breed. Being primarily grass-fed as opposed to a grain based diet, the meat is particularly nutritious.
Highland beef has lower levels of fat and cholesterol and a higher protein and iron content than other beef.
Who Knew? The British Royal Family maintain a purebred fold of Scottish Highland Cattle at Balmoral Castle where they are naturally raised and grass fed
It turned out that the reason the cows were no longer visible from the highway was to resolve a safety concern; Highway 28 is a very busy north/south cottage route. Too many people were stopping on the side of road to take photos of the cattle (they are pretty unique in this area), and the community was concerned about someone getting hurt.
Terry and Lisa were happy to allow me to take photos, too. Terry even called in the cows from another pasture so I could get some action shots as they ran to the fence. This also gave me the chance for some close ups! The lighting wasn't the greatest, but I managed to get several images I was pleased with.
On my request, Terry also led me into the pasture to pet one of the older cows, who recently bore a calf! I also took the opportunity to make some photos, of course.
Whether it's a horse or a cow or a goat (or even a chicken, probably), I experience a difficult to describe sensation when I am close to or physical touching these animals. A mixture of respect, reverence, awe (and a dash of fear, too, no doubt). These animals are just so large in comparison to the household pets I've lived with over the years. It is truly an memorable experience for this city-boy to have such close contact with these creatures.
This is one of the older cows. She had recently given birth. Terry led me to her and while he rested a hand on her back I took this photo. The twisted horns really add to her character and apparently just happen naturally with some of the animals.
In the couple weeks since posting these images, I'm also very pleased to share that one of my shots from Red Farm Lane (above) has sold.
For transparency, I did use Photoshop to remove his hand from the shot and to tidy up the field behind the cow's head. As the day very hazy, I also took the opportunity to add some subtle interest to the sky using Photoshop's Sky Replacement feature. And of course, I also spent time removing flies!
I’m looking forward to photographing this bunch again, perhaps in the winter.
Final Thoughts - The Ethical Use of Generative AI
As I've mentioned before, opinions on the use of GenAI are fairly polarized. I'm not a lawyer and I don't pretend to have all the answers, but I think it behooves both creators and companies supplying this technology to be ethical, responsible and transparent in the use of Generative AI. My good friend, Tom Green, discusses the importance of this self-regulation in a recent article on Medium, linked here. It is both sad and frightening how quickly human beings can knock things off the rails when it comes to technology.
From a "photographic perspective," my goal with Gen AI is to use it when there is no other satisfactory way to correct an element in an image in a timely manner, while keep the spirit of the original image in mind. This Muckle Coo project is an example.
From an illustrative/storytelling perspective, I'm more loose on my actions; I will add/remove elements to better tell a story.
Just as a I would have no second thoughts about using the Clone Stamp or Content Aware Fill, I will have the same regard for Generative AI; it is another tool in my arsenal.
While it is optional, I am a huge fan of the ability to enable Content Credentials in Photoshop, so that the changes to an image can be embedded in the file or in the cloud. These credentials help to establish provenance of an image and indicate how an image has been altered.
I hope you've found my process and workflow of interest and that you can see some of the practical applications Generative AI can have for photographers.
Until next time!
