[UPDATED] Explained: Under Display Cameras & its Application with the CEO of OTI Lumionics

UPDATE:

The second and final part of this interview which talks about transparent OLEDs and foldable has been published. You can check it out here.

 

Smartphone and TV displays have become incredibly sharp, bright, and contrasty over the past few years and a lot of this credit goes to the rapid improvement in OLED technology. This display technology has advanced to a level where we have flexible, foldable, and even rollable OLEDs in the market. 

But what’s next?

To talk about the future of OLEDs, we needed someone from the inside, an expert who’s actively involved in the development of OLED panels. Last week, I had the opportunity to interview the CEO of OTI Lumionics, Michael Helander. 

Michael has a Ph.D. in Materials Engineering from the University of Toronto. He along with his colleague Zhibin Wang cofounded OTI Lumionics back in 2011 after they made an important discovery that reduced the power consumption of OLEDs while increasing its brightness. 

Since then, OTI Lumionics has been fully involved in the research and development of OLEDs. But their most exciting development is the new ConducTorr™ Electrode that brings out new applications for OLED panels. 

In my interview last week, Micheal shared his incredible insights about under-display cameras and OLEDs. I learned so much about OLEDs in the interview that even after removing the non-important parts, this article was turning out to be exceptionally long. So, I’ve decided to split this interview story into two parts.

Today’s article will focus on under-display cameras, how they work, its future, and the challenges faced by manufacturers in mass production. 

The second part of this interview piece will be published later this week and will focus on transparent OLED TVs and its application, along with a few insights on foldable OLED panels. 

Please note that due to the nature of the interview, this article won’t be in a simple question and answer format. Instead, it will focus on specific areas of an under-screen camera like its development and applications. But rest assured, I have tried my level best to retain Michael’s tone and explanations. 

I hope this interview turns out to be incredibly insightful and gives you a thorough understanding of how under-screen cameras work and its future applications.  

 

How does OTI Lumionics contribute to the OLED industry?

Most of us have probably never heard of the name OTI Lumionics. This is because it doesn’t make consumer-grade OLEDs like LG Display or BOE. Instead, the company works behind the scenes and helps display makers to manufacture their OLED panels for smartphones, TVs, and more.

When asked about OTI’s contribution’s in the OLED scene, Michael explained that OTI Lumionics supplied advanced materials and chemicals to companies to manufacture high-quality OLED panels on a large scale. The company works in synergy with other suppliers like Universal Display Corporation to enable display makers to manufacture panels with higher efficiency, higher brightness, and longer battery life. 

The company has a dedicated Material’s Discovery Team which applies state of the art high-performance computing (HPC) simulations along with machine learning AI algorithms to discover and design production-ready advanced materials.

Their ConducTorr™Electrode for OLED Cathode Patterning, which enables better under-display cameras, is a product of this department. 

 

What is OTI Lumionic’s Conductorr™ Cathode Patterning Material (CPM)?

In simple terms, ConducTorr™ CPM enables manufacturers to pattern holes in between the OLED pixels for better transmission of light for the cameras underneath. 

To get a better understanding of how under-display cameras work on smartphones and other devices, we need to first understand how camera sensors work and why putting it under a regular OLED panel makes it almost impossible to reproduce images. 

Our smartphone camera sensors need light to capture images. This means, every camera sensor, including cameras that are hidden under the display, need access to light to process and churn out good quality selfies. But our regular OLED panels are not transparent, which means, it won’t allow light to reach the camera sensor that lies below it. 

So, in order for the under-display camera to work, you need the display or at least the section above the camera sensor to be relatively transparent and allow light and other signals (like for 3D face recognition) to pass through. This means, the OLED panels used in under-display camera phones have to be customized for this purpose – which is not an easy job, especially when you have to come up with solutions that are scalable for manufacturing millions of such panels. 

Michael explains, “So if you look at the cross-section of an OLED display, it’s made up of a bunch of different layers, kind of stacked one on top of the other. And at the very top of the display is actually one of the electrodes that’s used to get electricity into the display that’s been converted into light for the direct admission from each OLED pixel. And so, that layer of metal covers the entire surface of the display and it’s very thin. It’s like nanometer-scale thickness.”

He adds, “But because it’s metal, it still blocks a bunch of light, which lowers the brightness. If you’re trying to put a camera under there, it lowers the transparency and could create a bunch of problems. The challenge has been in manufacturing on a very large scale the way that these displays are made. There’s no way to effectively pattern that metal layer to create holes or make it thinner in some areas and thicker in others.”

This is the reason why we didn’t see under-display cameras until 2020. But now, companies have finally managed to improve the transparency of OLED panels for this purpose. And thanks to new solutions from companies like OTI Lumionics, increasing the transparency of a part of the display or the entire OLED panel is not so difficult anymore. 

OTI Lumionic’s ConducTorr™ CPM solution increases the transparency of the display during the manufacturing process by patterning holes in between the pixels.

Michael explains, “So then where we fit in, is our ConducTorr cathode patterning materials  (CPM.) You can think of it like an anti-stick coating for metal. So what display manufacturers do is they’ll deposit our material in areas [that] they don’t want any metal to form and then they do the rest of their normal manufacturing process. And they’ll end up with either holes in that metal layer, or some areas can be made thicker [or] thinner. It allows them to do a very high degree of optimization that otherwise is impossible to do in the existing manufacturing process.” 

The idea here is to use the space between the pixels, pattern holes in them, and thereby increase the transmission of light and signals. This is similar to the explanation given by Xiaomi on how its own under-display camera was developed (but without the holes). 

However, Michael adds that the quality and durability of OLEDs panels are considerably affected by the method used to make holes in these pixels. ZTE, Xiaomi, and OTI Lumionics essentially rely on using the space between the pixels to increase light transmission, but the way they achieve it can significantly impact the overall quality of the OLED display panel on top. 

Michael explained that none of the technologies in use right now, may it be ZTE’s under-screen camera solution or even Xiaomi’s solution, can match up to real-world expectations. On the other hand, he was confident that OTI’s ConducTorr™ CPM technology would enable better and more efficient under-display cameras. 

To further understand the difference in the approach adopted by different manufacturers, let’s take a closer look at the existing solutions from ZTE and Xiaomi. 

 

ZTE’s Under Display Camera Solution 

Instead of patterning (or making holes in) the top metal layer of the OLED display on the Axon 20 5G (which was announced earlier this month), ZTE decided to replace the top portion with a material that’s more transparent. So the camera that’s located on the top part of the display gets more light due to the higher transparency.

However, in the process, ZTE Axon 20 5G’s display on top of the under-display camera has a lower resolution (so the pixels are more spaced out, which means more space for light to pass). But to ensure that the top portion that features a lower resolution doesn’t stand out significantly, ZTE has to drive the pixels in that area way brighter than the rest of the display. This can potentially affect the reliability of the display area on top of the camera. After all, we all know how OLEDs are susceptible to burn-in. 

Another issue with ZTE’s solution is that the color quality of the display won’t be uniform. Since the part above the camera has a different, thinner, and more transparent material than the rest of the display, you’ll notice the slight shift in colors. 

So to get around this drawback, Micheal said that ZTE had to limit the UI to only show notifications and non-important content in the area where the camera lies so that it’s less noticeable to the user. But for different user interfaces and operating systems, this software solution is never going to be sustainable in the long run. 

While we haven’t yet tested the ZTE Axon 20 5G, this difference in quality was clearly highlighted by one of the reviewers (Pheonix Lab) from China.

READ MORE: Xiaomi Mi TV LUX OLED Transparent Edition is the cheapest transparent TV at 49,999 yuan ($7,195)

Xiaomi’s Under Display Camera Solution

Xiaomi has taken a slightly different approach to this problem. According to the publicly available information, Xiaomi decided to make the pixels in the area above the camera smaller (instead of reducing the resolution) than the rest of the panel. This creates more space between the pixels, allowing more light to pass through. It looks like a slightly better solution than ZTE but it has very similar drawbacks. 

Michael explained, “And so [reducing pixel size] creates more space between the pixels for light to go through but [Xiaomi] didn’t actually cut holes. So it’s still kind of limited in terms of quality and the problem with making a pixel smaller is…you have to drive them much brighter and so there could be potential burn-in issues.” He adds that there might also be a visible color shift in that area. 

In one hands-on video of the Xiaomi prototype (via PCHome), we couldn’t find any visible difference in the quality of the display in the area (top left) above the camera and the rest of the panel. So Xiaomi’s solution seems to be better than ZTE. However, the phone was a prototype model and the hands-on video didn’t really focus on the UI elements on top of the camera or its selfie quality.

Michael points out that both Xiaomi and ZTE will probably have to rely heavily on post-processing to make the selfies look better. Xiaomi could also leverage its technology acquisition from Meitu 2018. However, not all markets prefer heavily processed, beautified selfies, which is why Michael thinks that it’s going to be difficult to use this solution in the long run. 

 

How is OTI Lumionic’s ConducTorr™ CPM technology different?

According to Michael, in order to fully explore the potential of under-display cameras and even transparent OLEDs, companies will have to make holes in their OLED displays to achieve a high level of transparency. 

But the idea of making holes in between the pixels sounds quite complicated and risky. So I had to ask Michael about the durability of OTI’s solution. OLEDs have their share of drawbacks like dead/stuck pixels, burn-in, and even color shifts. So there’s always the question of how making holes in the panel would affect its durability in the long term. 

But Michael seemed pretty confident in his technology. He explained, “With our technique, we’re using the same evaporation process that [companies] use to actually make the red green blue (RGB) pixels in the OLED display, but it’s all low temperature. [There’s] no damage to the displays and we can go to any resolution – all the way down to a very tiny pixel size which would be equivalent to like thousand pixels per inch, which is about double the resolution of a normal smartphone display. And then, in that case, all of our data and our customer data show no degradation in the display quality and no long-term reliability issues.” 

His explanation sounds pretty convincing, but of course, without a real-world OLED product manufactured using the ConducTorr™ CPM solution, we will have to wait and see the difference in the quality and reliability of these panels. 

Editor’s Pick: BOE targets shipment of 40 million flexible OLED this year

Under-Display Cameras are coming to Mid-range phones First 

Perfecting the under-screen camera technology will take some time and hence, companies will be hesitant to use it in an expensive model. 

Michael said that in the first couple of generations of under-display cameras, manufacturers will have to sacrifice something to get it to work. This also brings us to another major highlight of under-display cameras of 2021. Unlike popular opinion, Michael says that the first generation in-display cameras are likely going to be implemented inside mid-range smartphones. So if you were expecting to see it on a flagship in the first half of 2021, you are set for disappointment.  

Michael explained, “We‘ve seen it in the past where Samsung and others have played out [experimental technologies] in some of their mid-tier products first to see how the market accepts it before rolling out across the board.”

“Likely for the first two generations, we’ll have to sacrifice something. Either display [will be] slightly worse, or the camera will be slightly worse,” he adds. “What the early product lets us figure out is, which is more important to the consumer — hiding the camera and making the display perfect or keeping the camera quality as good as possible and if the display is not quite perfect, that’s not a big deal, because it’s still better than a notch.”

So it’s going to be a trial and error for the first few generations until manufacturers gather enough consumer feedback to prioritize what’s more important – camera quality or display quality.

The world’s first under-display camera turned out to be a $320 mid-range Axon 20 5G. So ZTE has set the trend of what’s to come in 2021. We have also heard rumors about Xiaomi’s new in-display camera phone being a mid-range device. So top manufacturers are unlikely to implement an under-display camera solution in their expensive flagships, where there’s no room for mistakes. 

But this also tells us that under-display cameras won’t be significantly expensive to mass-produce.

 

Under Display Cameras aren’t necessarily more expensive than a notch or punch hole solution

According to Michael, it’s still too early to comment on the exact pricing of OLED panels that support under-screen cameras but he is positive that the prices can come close to a traditional punch-hole or notch camera solution. 

“No one really knows what the final yield is going to be, and then the yield** directly drives what the panel prices are. One of the promising things is, of course, ZTE is putting [its UD camera panel] in a very low-end phone. It was like $300!”

(**Production yield for a panel manufacturer is the number of good display panels as a percentage of the total number of display panels produced.)

Michael adds, “Our expectation is that once you know the production is scaled, the cost for the panel should not be really that much different than the hole punch or notch displays of today. Because even though they have to add something to make the display semi-transparent they don’t have to cut a hole or cut a notch which has some potential yield loss. So the cost will kind of offset each other.” 

His explanation gives us an idea of how under-display cameras can become mainstream in the next few years – possibly as common as a notch or a punch-hole solution. This makes things very exciting, especially because of what comes next after UD cameras become mainstream.  

 

UD camera solution enables Larger Sensors, Multiple cameras, & Better Positioning

The obsession with a full-screen display experience may have driven us to the point of shifting cameras underneath and there are several advantages in this design. 

Currently, we are limited by the size of the notch or the punch hole on the display for the camera sensor. Imagine what happens when your camera can hide behind the screen, anywhere. 

When I asked Michael about the possibility of adding a larger sensor under the camera for better selfies, he was thrilled to say, “It’s a really great question and I can even take it one step further. We’ve actually seen a number of customers interested in making, potentially, the entire display transparent.”

So are we looking at completely transparent phones anytime soon?

Just as I was about to ask this question, he added that the screen will still look like a regular OLED panel but the benefit of making the entire display transparent is allowing manufacturers to be flexible with the placement of the under-display camera and sensors. A larger transparent area would enable manufacturers to add in a much larger sensor for selfies underneath. 

But that’s just one possibility. 

They can also add two or three cameras upfront with multiple IR sensors or even something like a Soli sensor without disturbing the full screen, distraction-free visual experience on phones. 

But it’s not just about the number of sensors inside, it’s also about their placement. Michael adds, “[Right] now, I’m limited to only [one or two cameras] at the top bezel. And maybe in terms of the internal design of the phone, that’s actually not the best place. If I could move it somewhere else, I’d have more space for the battery, or I could make the phone thinner or make the phone lighter”.

Another advantage of flexible camera placement on the front is the angle of view. He says that with under-display cameras, camera placement is no longer limited on the top bezel, whether it’s on a phone or even a laptop. This gives them the freedom to place it anywhere on the display like the center of the OLED panel which probably will allow for a more immersive video calling and camera experience. 

This is especially true in the case of larger devices such as laptops and TVs. Imagine if you could hide the camera at the center of their screens — it would allow for better quality video calls, especially if it feels like the person on the call is directly looking at you. In the case of existing solutions, if the person on a call is looking at the screen of the device, you’d often find that the camera focuses on his or her forehead, which doesn’t make for an immersive experience. This is particularly true in this age where video calls and video conferencing are becoming a vital part of our personal as well as professional lives. 

Talking about video conferencing, Michael says companies like Microsoft are even considering the potential of under-screen cameras inside large TVs that are part of a video conferencing system. He adds, “What if the TV is transparent and then I have a bunch of cameras behind it. And again, it’s much better usability in terms of video call quality and social interaction.”

 

My conversation with Michael made me realize that under-screen cameras are definitely the next step in smartphone evolution. The true purpose of spending millions of dollars and years in R&D of under-display cameras is not just to bring a full-screen experience to the user. But it is to enable manufacturers to be flexible with the front camera solution, thereby opening up a lot of potential new applications, that wouldn’t have been possible before. 

The first generation of under-screen cameras won’t be perfect. However, with innovative technologies like OTI Lumionics’ ConducTorr™ CPM, hopefully, within the next couple of years, we’ll meet phones that can match today’s flagships, in terms of both display and camera quality.

OTI recently received a strategic investment from LG Ventures (the investment arm of LG Group) to help accelerate the commercial adoption of its materials and scale up operations. The latest version of OTI’s ConducTorr™ CPM is ready for mass production and will ship to smartphone display customers in Q4 2020. And the first set of under-display camera phones manufactured using this material is expected to hit the markets in 2021. 

In the next article, coming up next week, you’ll get to read more about transparent OLEDs, their applications, and foldables. So stay tuned to Gizmochina!

UP NEXT: China becomes the largest buyer of smartphone OLED panels: UBI Research