Discover Sea-Thru: The Future of Underwater Photography

You would have seen images or videos that are taken underwater, which, even when they are blurry and different from their original colour, are beautiful. Imagine if you get to see them clearly and that too in their colours!!! Now, we can bring that imagination to reality, with Sea-Thru, an algorithm or a process that can remove water from underwater images to produce clearer and more exact pictures. 

In this blog, we will look in detail at the Sea-thru algorithm and the physics behind this. Also, we will get into detail how the Sea-thru algorithm removes colors from the images, what it is dependent on, the pros and cons of the Sea-thru algorithm with example images. Let’s get started.

What is “Sea-thru”?

Sea Thru is a physics-based image enhancement tool that is specifically designed to ameliorate underwater photography. Color cast (green or blue fog) can be removed from the underwater shots to restore the true beauty of visually enriched underwater scenes by the dependent model Sea-thru. This aids oceanographers and marine biologists in comprehending the underwater ecosystem.

Are you interested in learning more about time series analysis? Enroll in Guvi’s IITM Pravartak certified Artificial Intelligence and Machine Learning Course. This covers all the important concepts of artificial intelligence from basics such as the history of AI, Python programming, to time series analysis, deep learning, and image processing techniques with hands-on projects.  

Why is Sea-thru “physics-based”?

To understand this, we need to have a basic understanding of how light interacts with matter. 

Have you ever wondered why the sky looks different at specific times of the day, or why the sky is colored? 

When light enters the atmosphere, many things can happen. For context: 

1. Transmission – Light passes through matter without interaction. 
2. Absorption – Light is absorbed by matter, and the energy helps the electron (which is still a small particle inside a minuscule atom) get excited. 
3. Scattering – When light interacts with matter, it can be redirected in different directions. When a photon interacts with an electron, the electron absorbs the energy and gets excited. The electron then gives back the same light that was absorbed after a very small interval of time. 

Recall the colors that make up the rainbow. It is the composition that makes white light. When you let white light enter a prism, it will come back as seven different colors (as demonstrated by Newton’s prism). The colors are arranged in order of decreasing energy: VIBGYOR, with violet, indigo, blue, and green categorized as higher energy photons and the rest as lower energy photons. Higher energy photons scatter more, while lower energy photons transmit more.

In the morning and evening, the sky appears as a mixture of red, orange, and yellow. This is because the light from the sun needs to travel a greater distance to reach you. As the Earth rotates, in the morning, your location is gradually exposed to the sun, and the light takes longer to reach the slightly exposed part compared to the directly exposed part. Blue photons of sunlight undergo so much scattering in the atmosphere that they are lost along the way. However, red, orange, and yellow photons can transmit (meaning they can travel a long distance without interacting with molecules). This is the reason why the morning and evening sky appears red.

During the daytime, the exposed regions have access to more blue light emitted in all different directions, making the sky appear blue. This phenomenon is called Rayleigh scattering

Scattering is the same reason why underwater looks green or blue (Colorcast). When light enters the water, it gets absorbed and scattered by the water molecules, and this effect is more pronounced for shorter-wavelength colors like blue and green. This scattering causes the blue and green light to bounce around and spread out in all directions, resulting in a greenish-blue color cast in the underwater environment. In contrast, longer-wavelength colors like red and yellow are less affected by scattering and can penetrate deeper into the water, which is why these colors are less visible in underwater environments. Though red transmits more but the red changes into a brown or dark hue in water.

Sea-thru is physics-based because it restores the true color by removing color cast, which is based on the study of light propagation in water.

How does SeaThru remove the color cast, and what model is it dependent on?

The previous model, designed for image transformation in foggy land environments, did not consider the different optical properties and the fact that light’s velocity is different in water. As a result, important information is lost in underwater images with fog, making it difficult for machine learning models to accurately identify objects.

Sea-thru was inspired by the concept of radiance or brightness of light as a function of depth in the water, and it addresses the limitations of the previous model by taking into account the unique optical properties of underwater environments.

In simpler terms, attenuation refers to the decrease in the amount of light as it travels through water. To correct the distortion caused by this, we need to first calculate the original form of light before it enters the water and then use this information to estimate the attenuation coefficients, which can help us create depth maps. These depth maps can then be used to recover the true colors of the scene.

The equation J * exp(-betacz) + B0 * (1 - exp(-betacz)) is used to estimate the attenuation, where the term exp(-betacz) accounts for absorption and (1 - exp(-betacz)) accounts for scattering. However, this equation does not fully account for the behavior of light in water, and the attenuation constant is only wavelength-dependent, which limits its accuracy. As a result, a revised model was introduced to better account for the complex optical properties of water.

In this revised model, both constants are different, helping us to capture the following non-optical properties.

• Camera sensitivity
• Illumination
• object reflectance.

Now understanding the model behind sea-thru, let’s understand what the sea-thru algorithm does.

Sea-thru uses a deep learning algorithm to estimate the attenuation constant. The algorithm is trained on a large dataset of underwater images with known ground truth values for the attenuation constant. The Sea-thru algorithm uses the revised model to estimate the attenuation coefficients in underwater images. These coefficients are then used to create depth maps and correct the distortion caused by attenuation, thus recovering the true colors of the scene.

To achieve this, Sea-thru first estimates the attenuation coefficients from the input image by solving an optimization problem using the revised model. It then uses the estimated coefficients to generate a depth map, which indicates the distance of each pixel in the image from the camera.

Finally, Sea-thru applies the estimated coefficients and depth map to the input image to correct for attenuation and recover the original colors of the scene.

Advantages and Disadvantages of sea-thru

Advantages of Sea-Thru:

1. Recovering True Colors: Sea-Thru is able to recover the true colors of underwater scenes by correcting the distortion caused by underwater attenuation.
2. Depth Mapping: The algorithm creates accurate depth maps, which can be useful for applications such as underwater mapping and navigation.
3. Non-Destructive: Unlike traditional underwater photography techniques that require the use of artificial lighting, Sea-Thru can be used with ambient light, which is less intrusive to marine life.

Disadvantages of Sea-Thru:

1. Limited to Clear Water: Sea-Thru is designed to work in clear water environments and may not be effective in murky or highly turbid water.
2. Requires Accurate Depth Estimates: The accuracy of Sea-Thru is highly dependent on accurate depth estimates, which can be challenging to obtain in some underwater environments.
3. Time Consuming: The processing time required to generate accurate results with Sea-Thru can be significant, making it less practical for real-time applications.

Images before and after sea-through

Conclusion

Sea-Thru is a powerful tool that has the potential to revolutionize the field of underwater imaging. By removing water from underwater images, the software can produce clearer, soulful images that reveal the hidden beauty of the marine environment. Whether you are a marine biologist, an underwater photographer, or simply someone who enjoys exploring the ocean, Sea-Thru is worth checking out.

Reference:

(2015). Re: Can a photon with more energy than the energy level difference be effectively absorbed?. Retrieved from: 

• https://www.researchgate.net/post/Can_a_photon_with_more_energy_than_energy_level_difference_be_effectively_absorbed/54fad49bd767a64d788b461b/citation/download.
• https://openaccess.thecvf.com/content_CVPR_2019/papers/Akkaynak_Sea-Thru_A_Method_for_Removing_Water_From_Underwater_Images_CVPR_2019_paper.pdf