I made this, fun to see my project on the front page!
The main question I tend to get from hacker-types is "what are the tech specs?". The demos run on a Mac with custom software I built for projection mapping/rendering/etc. (the hand tracking uses Apple's built-in framework, which is very good), a small consumer 4k laser projector, and a basic webcam for vision.
But really, this is primarily a research/design prototype - I think it's much more important to figure out what the user experience should ideally feel like, if the interactions are even plausible & desirable in the first place; and then work backwards to the technology.
Is a basic webcam really able to read text at that angle for the “what is this word” example?
I wasn’t sure if any of this was real and that example looked extremely fake to me because I had a hard time believing the device being able to read text from that distance.
In that demo, the lightbulb sits on the desk, less than a meter away from the page. What feels implausible about this setup? This is kind of thing you can do with your smartphone today.
So is the demo real or is it smoke and mirrors? Could you have pointed at other words and have it work?
(I think I overestimated how much of an angled shot it was the first time I saw it, on top of how easy or hard turning your finger into “actually capture this text”… but you’re totally right that in principle a camera at a non agressive angle can capture that text)
(I like this overall concept a lot! Cooking is the right example, voice interfacing’s 96% use case is basically “my hands are dirty but I must use computing”)
[−]TheOtherHobbes · 2026-10-11 Sun 10:04 UTC ·
link
It doesn't feel implausible, it is implausible, because your idea would require either a camera with good optics pointing in a specific direction, which means it's no longer a set-and-forget lightbulb, or some kind of 360 view more-or-less 3D camera to cover all the angles, which probably isn't going to have the resolution to read angled text at that distance.
At the very least it's going to have a range limit on that feature.
And there's a projector in there too.
In something smaller than a smart phone.
It's a great concept, but not a very workable one. Kitchens get steamy, tables can be huge, and management will want to use the cameras for user surveillance.
I met @heliographe a few years ago in a HN Tokyo (Slack) meeting. Happy to see you are still designing awesome things after your time at Apple. I look forward to seeing how your project(s) develop. PS: as soon as I saw the kitchen I knew this was Japan, and then I saw the hair and though —“hmm, I feel like I have seen this person before”— and it was you!!
Right. Tiny projectors have been done before.
Samsung had a smartphone with a projector back in 2014, but nobody wanted it.
Some startup called Moviphone has announced a new one for 2027.[1]
The use cases have not been compelling and fighting ambient lighting with a small projector is always tough.
Possible use case: projecting menus, ads, and bills on restaurant tables. With a camera and enough automated layout to work around what's on the table.
It might sell to the people who buy refrigerators with displays on the door.
Could be useful as a cooking training aid, like cooking videos but present where you need them.
The light bulb form factor is cute but probably not that useful. Most of the light will be aimed in non-useful directions. Look into ultra short throw projectors.
It's a good thing that most inventors don't fall for the "something vaguely like this has been tried with inferior technology before, and it sucked" line of reasoning, or our world would look very different :)
Well, if you're a inventor, you look at the prior art to see if it's been done before. I hold six issued patents, two of which were very profitable. I had one patent rejected because I missed relevant prior art from 1948.
Then again, I feel like I know industrial designers who play with building on ideas that came before, reexamining whether those ideas might have a fresh form and a place in the world today.
Who iterate and hone until they have something worth existing in today’s world—often doing do more publicly than corporations’ R&D departments—then let the patent technicalities be the well-paid lawyers’ problem…
> The light bulb form factor is cute but probably not that useful. Most of the light will be aimed in non-useful directions. Look into ultra short throw projectors.
gonna hard disagree! you aren't going to have 360° projector in there with wasted light going nowhere. you're going to have 1-4 steerable projectors. the "facade" as a globe bulb means you can place it at any direction you like depending on where the cord needs to go. for ceiling as well you don't want it to have a fixed throw direction. bulb makes perfect sense.
Imagine downvoting John effing Nagle. However my humble opinion is that sometimes revisiting, if not reinventing, the wheel is beneficial in light of accrued epistemic benefits (or for the fun of tinkering with a cutesy creation) . Not for patenting, of course. Besides, the bulb form-factor is not too different from cctv cam ceiling installations.
This is an awesome concept. How do you see this working across several rooms (if you've thought this far)? This would be great for when I have ideas in the shower.
The appeal of the lightbulb form factor is that you can mount it in a portable base to move it around the house; or in a fixed socket for broader room coverage.
It's not mentioned in the piece, but for the latter I don't see any technical obstacle to having several of these in different rooms in the house, or a couple in the same room to augment coverage.
A lot of people have Alexa or Google devices in each of their rooms, this seems like a very logical step. Perhaps 2 generations from what is being currently released based on how things go. Maybe sooner!
Could you do beamforming with this, or would that get too expensive/awkward? At the very least, having more of these guys in the house (if you can get the tech cheap, ofc) would lead to better coverage potentially?
The main issue is probably gonna be that the display sucks compared to actual displays :( but it looks cool though! and the whole VLM thing makes a lot of sense
Its sad that search engines have become useless. There was another UX researcher who did something very much like this and then later got recruited to Google. As I recall he was using the cameras in Wii Nunchucks because they could do SLAM easily. And yet I cannot find his research, sigh.
The main question I tend to get from hacker-types is "what are the tech specs?". The demos run on a Mac with custom software I built for projection mapping/rendering/etc. (the hand tracking uses Apple's built-in framework, which is very good), a small consumer 4k laser projector, and a basic webcam for vision.
But really, this is primarily a research/design prototype - I think it's much more important to figure out what the user experience should ideally feel like, if the interactions are even plausible & desirable in the first place; and then work backwards to the technology.
I wasn’t sure if any of this was real and that example looked extremely fake to me because I had a hard time believing the device being able to read text from that distance.
(I think I overestimated how much of an angled shot it was the first time I saw it, on top of how easy or hard turning your finger into “actually capture this text”… but you’re totally right that in principle a camera at a non agressive angle can capture that text)
(I like this overall concept a lot! Cooking is the right example, voice interfacing’s 96% use case is basically “my hands are dirty but I must use computing”)
At the very least it's going to have a range limit on that feature.
And there's a projector in there too.
In something smaller than a smart phone.
It's a great concept, but not a very workable one. Kitchens get steamy, tables can be huge, and management will want to use the cameras for user surveillance.
Possible use case: projecting menus, ads, and bills on restaurant tables. With a camera and enough automated layout to work around what's on the table.
It might sell to the people who buy refrigerators with displays on the door. Could be useful as a cooking training aid, like cooking videos but present where you need them.
The light bulb form factor is cute but probably not that useful. Most of the light will be aimed in non-useful directions. Look into ultra short throw projectors.
[1] https://moviphones.com/
Who iterate and hone until they have something worth existing in today’s world—often doing do more publicly than corporations’ R&D departments—then let the patent technicalities be the well-paid lawyers’ problem…
Nothing truly new under the sun
gonna hard disagree! you aren't going to have 360° projector in there with wasted light going nowhere. you're going to have 1-4 steerable projectors. the "facade" as a globe bulb means you can place it at any direction you like depending on where the cord needs to go. for ceiling as well you don't want it to have a fixed throw direction. bulb makes perfect sense.
It's not mentioned in the piece, but for the latter I don't see any technical obstacle to having several of these in different rooms in the house, or a couple in the same room to augment coverage.
How many watts does the projector use for how many lumens? I would be curious if these numbers plus size would be a fundamental limit to the tech.
The main issue is probably gonna be that the display sucks compared to actual displays :( but it looks cool though! and the whole VLM thing makes a lot of sense
http://johnnylee.net/projects/wii/