Facebook has officially launched a new open source image processing library called Spectrum, which the company said has already helped improve the reliability and quality of images uploaded through its own suite of apps.
The social networking giant first unveiled Spectrum to the public in November, and it has been available in beta (v0.9) since that time. Today heralds the official launch of Spectrum 1.0.0 on GitHub, as Facebook strives to improve the technology by opening it to the developer community.
Cameras are now pretty much the focal point in any new smartphone launch, with nifty AI-powered software serving as selling points and camera sensors becoming increasingly more powerful. While this is great for mobile photographers, the technological implications of this from a social sharing perspective are significant.
Large image files not only consume more memory on a device, they also consume more network bandwidth when shared online — which is why apps such as WhatsApp and Facebook compress images. Platforms like these automatically resize and reformat photos, ensuring minimal stutter during upload and optimal loading time on the other side. But there is a trade-off in terms of image quality — what was a 3MB picture at 2980 x 2384 pixel resolution could be roughly a fifth that size when displayed in the app, which translates to reduced clarity.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
This is the challenge for any app that allows image uploads where speed and reliability are paramount: How can it ensure maximum quality without compromising on the upload experience? This is where Spectrum comes into play.
On the spectrum
Spectrum is a client-side image transcoding library for both Android and iOS apps. In the first stage, it works to reduce the file size, which in turn means faster uploads (and less mobile data consumption). Using a “declarative” API, Spectrum seeks to make it easier for app developers to control the image quality without having to worry about the individual steps required to get there, which might have otherwise involved writing additional code to leverage other compression libraries, for example.
In short, rather than telling an app step by step how an image should be transcoded, Spectrum allows developers to stipulate what they want done — and Spectrum takes care of the orchestration.
According to Facebook, Spectrum prefers a lossless approach when cropping and rotating JPEG images, while in resizing it “optimizes the interplay between decoder sampling and pixel-perfect resizing.” Spectrum integrates with native image compression libraries, including MozJpeg, a JPEG encoder launched by Mozilla’s research team back in 2014, which can reduce a file size by 10-15 percent in preparation for upload.
This also enables control over “more advanced parameters,” such as chroma subsampling, which is a compression practice that attributes less resolution to an image’s color in favor of luminance data. However, certain kinds of images may require more defined colors, particularly those involving sharp edges or illustrations, which is where Spectrum intervenes. “The consistent API makes these features accessible to developers who are not image experts,” Facebook said.
Moreover, given that Spectrum is open source, anyone can add support for new codecs via plugins and debug to their heart’s content.
Facebook told VentureBeat that it started development on Spectrum around 18 months ago and has gradually incorporated it into its own apps. During the 0.9 prerelease phase, Facebook gathered input from the developer community in preparation for today’s official rollout, which incorporates a number of fixes and “support for less common” chroma subsampling in JPEG files.
