What makes a good picture? Is it color? Composition? If you ask Microsoft, they might say pixels.
Its certainly the approach evident in their new iPhone photo app, Microsoft Pix, which takes 10 frames and then looks for, among other things, the best pixels from all of them to make the best possible picture.
Available starting today in the Apple App Store, Microsoft Pix aims to bring sophisticated and automated photo processing to every iPhone from the iPhone 5S on up. The app, which Ive been testing for a week, doesnt add megapixels, but it does use a fair amount of Microsoft Research artificial intelligence to choose pixels, and identify elements that not only make the best picture, but ones that can create and play back cinemagraph images without the need for Apples latest iSight camera or 3D touch hardware.
The free app does not try to replace Apples Camera app. Instead, you have to choose to open and use it (a bigger hurdle than one might think). The relatively spare interface offers the option of shooting photo or video and a big shutter button. Hit the button and the app takes a shot.
However, its not that simple. Microsoft Pix is designed to get the photos you miss. As soon as you open the app, its buffering frames. According to Microsoft, this is because people often miss a great shot because they see it, lift the phone to take a picture and then hit the shutter, often a second or two after the optimal moment has passed.
This may or may not be true. I capture a lot of great moments on my iPhone, even with that so-called delay, but I get Microsofts intent. Capturing the frames before and after someone takes a picture gives Microsoft Pix a lot frame options.
It also applies a bunch of photo-enhancing filters that look at contrast, tone, white balance and shadows. To reduce noise on low-light shots, it pulls better pixels from some of the other frames.
I took a lot of photos with both an iPhone 6 loaded with the app and compared to the default photo app and with a pair of iPhone 6Ss one had the app and one didnt (Microsoft even lent me a rig that could fire off both phone camera apps at once).
The photos from Microsoft Pix looked consistently better. The images were cleaner, brighter and often better composed (the eyes were open, the action shot looks less awkward and more exciting). It auto-adjusted back-lit photos, but with more finesse than Ive seen in other image-enhancement apps. Instead of just brightening the foreground and losing the detail in the back, it managed to improve the foreground image while maintaining the overall image contrast.
In general, the Microsoft Pix algorithm removes a layer of softness I’d almost call it haze from low light images and makes the colors in good lightning really pop. There’s also no perceptible loss of information, though the image might appear a bit darker or more contrasty. In some low light situations, however, Microsoft Pix can turn the warm light a little cold and almost bluish. It’s not distracting, but it might help if Microsoft included some sort of algorithm strength slider for times when you think it’s done too much.
The other issue is how long, in the beta I tested, it can take to process each image. In my experience it can take up to 3 or 4 seconds. That may be solved in the final app.
Microsoft Pix did not auto adjust every image. On occasion, I got a photo that offered no compare button and appeared virtually the same as my original iPhone 6 or 6S shot. This was usually in situations where there was more than adequate lighting and no movement to capture.
After you take each photo, a little review button with a tiny image thumbnail appears. You have to click that to see your work. The result is the best image according to Microsoft Pixs image processing algorithm. Theres a faint Compare button on top of the image that you can press to review the actual photo you captured; the originals honestly never looked quite as good (most other people I showed these images to agreed with my assessment). Sometimes the app will present you with more than one Best Photo option and you can then choose your favorite. All of these photos, by the way, are saved to your standard camera roll.
As a bonus, Microsoft Pix is also looking for interesting motion in every shot and can automatically create live, cinemagraph-style images. I was often surprised by the images Microsoft Pix turned into moving images: It simply noticed interesting movement that my eyes missed. In one shot, it kept the background perfectly still, while the hair and outfits on both my subjects fluttered in the wind. In another, a group of coworkers were gesturing while watching the start of the Democratic National Convention.
If you like the Live Image, you can export the four-second, endlessly looping video and post it anyplace that accepts a standard iPhone video file or you can convert it into a GIF using a third party app, as I did with mine.
There are a number of significant differences between Microsoft Pixs Live Images and Apples Live Photos. Only the iPhone 6S and 6S Plus can create Apple Live Photos. Thats because the iPhone does most of its image processing in hardware. Microsoft Pix does all its work in software, which is one of the reasons it can work with older iPhones.
Apple Live Photos basically capture a few frames before and after your photo, turning the whole thing into a video. Microsoft Pix is using Microsoft Research technology originally designed to create cinemagraphs, programming that specifically looks to separate interesting movement from good static imagery (a nice portrait in the foreground and a moving waterfall in the background) and to combine the two into a visually interesting whole.
Good stuff, but
Microsoft Pix also also blends in hyperlapse technology, but instead of forcing you to shoot in time-lapse mode, it allows you to shoot regular video (or import it) and convert to hyperlapse. You can then adjust the playback speed. I honestly didnt see much use for this feature.
The app is designed to identify better facial expressions for better portraits and group shots. I didnt always see this; both the shots in the app and my iPhone Camera shots looked decent. Perhaps I was being too careful. So I tried some selfies (Microsoft Pix works with either iPhone camera) and purposely blinked my eyes a few times while taking the shots. The results werent great.
Overall, Microsoft Pix is an impressive display of Microsofts programming prowess and, in most instances, especially for group photos, action and cinemagraphs, produces better photos than the default iPhone Camera app. However, its integration with the Apple ecosystem can only go so far and as long as people are swiping up (or, soon with iOS 10, to the left) to access the iPhone camera, I do not see them making much headway with this smartly executed app.
Made iPhone photos noticeably better Smart Easy to use Adds live photos capabilities to older iPhones
Slow in pre- release
The Bottom Line
Microsoft’s programming prowess is clearly on display in Microsoft Pix and it does create better iPhone photos, but how do they train people to use it instead of the default camera app?