Apple patent application describes Apple Pencil that can sample real-world colors and textures

Apple has filed a patent for an Apple Pencil stylus that can identify and digitally replicate colors and textures from real-world objects. Patently Apple spotted the application last week (which we saw via MacRumors), which describes a stylus equipped with a color sensor, light detector, and light emitter within its “pencil” tip. According to the patent doc, the stylus can wirelessly transmit “color sensor measurements, texture measurements, and associated angular orientation measurements” to external devices like an Apple iPad. 

The patent says that the stylus is capable of measuring texture by detecting how lighting and angles affect an object’s surface. Similarly, a combination of optical sensors can be used to measure how reflective an object is and provide more accurate color information.

The concept of a stylus or pen that can sample colors from the real world isn’t new. The Scribble pen was released in 2014 and is able to scan and reproduce colors both in real ink and as digital swatches, while sensors from brands like Nix and Palette Pico are used to match real-world shades with digital color systems like sRGB, CMYK, and HEX codes. These sensors can be used to replicate long-forgotten paint formulations and for other practical applications.

As with all patents and patent applications, there’s no guarantee this concept will become a real Apple product, but I’m keeping my fingers crossed regardless — not just because of the possibilities of color sampling with an Apple Pencil but because a stylus that can accurately reproduce textures would make my 3D sculpting hobby a whole lot easier. Physical textures can be tedious to design from scratch, and there aren’t always suitable options readily available in applications like Unity or ZBrush. For content creators who create digital assets for video games or models for 3D printing, being able to quickly and easily copy real-world textures on an iPad could be a game-changer.

A souped-up Apple Pencil would also make sense given the iPad’s increasing popularity with digital artists and graphic designers. Applications like Procreate and Clip Studio Paint have been warmly received on iPadOS, and devices like the standard iPad and iPad Air are relatively affordable compared to the cost of drawing tablets from brands like Wacom or Huion (especially if you also factor in the laptop or desktop computer needed to run them). The iPad Pro, meanwhile, is capable of running illustration software close to, if not on par with, many equivalent desktop graphic design apps. And when combined with the Apple Pencil, the iPad is incredibly lightweight and portable compared to tablet workstations.

The iPad hasn’t made as much of a splash with 3D artists as it has with illustrators and graphic designers due to the compute demands of 3D software. But improvements are being made thick and fast with each iPad generation. I use iPadOS apps like Forger and Nomad just as much as I use Blender on my desktop PC these days, for example, both of which run perfectly on my 2019 iPad Pro. As demand for 3D assets increases (thanks to the developing metaverse, 3D printers, and video games) iPads and Apple Pencils could soon become a more practical tool for digital sculpting professionals and students alike.

It’s worth remembering, of course, that Apple files a lot of patents each year, many of which never become actual products. In fact, Apple previously filed patents for a texture-detecting stylus in 2015 and a color-swatching Apple Pencil in 2020 — neither of which have been realized yet. Still, given the current-gen Apple Pencil 2 was released back in 2018, it certainly seems as if the device is due an update. If Apple does end up releasing a combined color- / texture-detecting Apple Pencil, I’ll be the first in line to purchase it.