The iPhone 11's Deep Fusion camera is now in the iOS 13 developer beta

Apple's photography system Deep Fusion has arrived on the latest developer beta for iOS 13, and hopefully hinted that it will be shipped to the iPhone 11 and 11 Pro soon.

To update your memory, Deep Fusion is a new imaging pipeline for medium-light images, which Apple senior VP Phil Schiller called "computational photography mad science" when he introduced it on stage. But like much of iOS 13, Deep Fusion wasn't ready when the phones came two weeks ago. And while the iPhone 11 and 11 Pro have extremely impressive cameras, Deep Fusion's is meant to offer a huge step forward in indoor and mid-light situations. And since so many pictures are taken indoors and in medium light, we can't wait to test it. Here is a sample shot shared by Apple:

  A Deep Fusion image of a woman in a sweater

Apple loves to showcase Deep Fusion with images of people in sweaters.

With Deep Fusion, iPhone 11 and 11 Pro cameras have three modes of operation that automatically kick in based on light levels and the lens you use:

  • The standard wide-angle lens will use Apple's enhanced Smart HDR for light to medium light scenes, with Deep Fusion as kicks in for medium to low light, and night mode comes for dark scenes.
  • The telephoto lens will largely use Deep Fusion, with Smart HDR taking over only for very bright scenes, and Night mode for very dark scenes.
  • ultrawide will always use Smart HDR, as it does not support either Deep Fusion or Night mode.

Unlike Night mode, which has an on-screen indicator and can be turned off, Deep Fusion is completely invisible to the user. There are no indicators in the camera app or in the photo role and it does not appear in the EXIF ​​data. Apple tells me that it is very intentional, as it does not want people to think about how to get the best picture. The idea is that the camera will do just that for you.

But in the background, Deep Fusion works quite a lot of work and works a lot differently than Smart HDR. Here's the basic breakdown:

  1. When you press the shutter button, the camera has already taken three pictures at a fast shutter speed to freeze motion in the shot. When you press the shutter button, it takes three extra shots and then a longer exposure shot to capture details.
  2. The three common long exposure shots and shots are merged until what Apple calls a "synthetic long". This is a big difference from the Smart HDR.
  3. Deep Fusion selects the short exposure image with the most detail and merges it with the synthetic long exposure. Unlike Smart HDR, Deep Fusion only merges these two frames, no more. These two images are also processed for noise other than Smart HDR, in a way that is better for Deep Fusion.
  4. The images run through four detail processing steps, pixel by pixel, each tailored for increasing amounts of detail – the sky and the walls are in the lowest band, while skin, hair, textiles and so on are the highest level. This generates a series of weightings for how to mix the two images – taking details from one and tone, color and brightness from the other.
  5. The final image is generated.

All this takes a chin longer than normal Smart HDR image – somewhere around a second in total. So if you take a bunch of photos and jump immediately into the camera roll, you will first see a proxy image while Deep Fusion is running in the background, and then it will jump to the final version in more detail, a process Apple says should not take more than a quarter to a half second by switching to the camera roll.

But all this means that Deep Fusion will not work in burst mode. You will notice that the burst mode itself has been emphasized through the Camera app in iOS 13, since all these new modes require the camera to take multiple exposures and merge them, and Apple's new hold-to-take video mode is a bit more useful anyway.

Here's another Deep Fusion image of a beautiful person in an Apple sweater. It is absolutely impressive. But we need to see how Deep Fusion works in practice when people catch up with developer beta. If it's as impressive as Apple claims, the iPhone 11 camera will jump even further ahead of the current competition, setting a high bar for Google's upcoming Pixel 4 to remove.

  A Deep Fusion Shot of Another Person in a Sweater

Deep Fusion: Sweater Mode

Source link

Back to top button