Cage Against the Machine
The story of writing Nic or Not in React Native
Nic or Not is a real-time face recognition app that can identify one particular person from a crowd. That one person is Nicolas Cage. I got to present this at a tech conference a month ago.
Why would I anyone do such a thing? Here’s the story of the app Nic or Not
Everyone saw “Not Hotdog” on Silicon Valley, and we laughed with the rest of the world, but when I saw it was a real app and written in React Native, I didn’t believe it. Reading the blog post on the process of “Not Hotdog” and even Tensorflow; I found out a hard truth. I knew nothing about any of the technology they used. For a little while, it was easy to concede that Machine Learning (ML) and AI were just not for everyone, and so I moved on… right?
🤔… 🤔… 🤔…
After some time, and some exposure, I found myself staring at the same puzzle, pondering how it all works. Like a half-read book or season cliffhanger, my subconscious begged to understand this mystical algorithm.
With each video, reference, or talk I saw, I became more entranced with the world of ML. Passion is the ultimate gift in tech craft.
Think about it… there’s a new world of algorithms no person could ever write. Don’t you want to know how something like that is made?
I researched, I completed the famous Andrew Ng Coursera course on ML, and I blogged about this passion. Today that blog post serves as my most popular post to date. I highly recommend you earmark it for a read, as it might inspire you, too.
Once plugged in, I started seeing things; new things in software I couldn’t believe. Rather than understanding AI, I was exposed to a new set of questions and solutions. People creating strange yet exquisite operations I hadn’t imagined. Around every corner of ML there was a new paradigm-shattering masterpiece. I started collecting the fun ones into a twitter account I deemed “Fun ML”
With the knowledge I was gaining, my brain quickly started coming up with different and magnificent products, algorithms, and ideas. Ideas beyond my ken. I realized I needed to finally get my hands dirty and start working on something to actually see it for myself.
Why Nic or Not?
As a newcomer, I needed something exciting, achievable, mobile-related, and well… something just entertaining to start with. I work in React Native with the good folks at Infinite Red, so my audience isn’t academic; it’s nerds who ship. Bringing silly ML to the phone was the perfect union of all the above while raising the bar. It wouldn’t do to simply take a photo like “Not Hotdog”, I had to attain something new. Real-time facial recognition of multiple people was a new and attractive level.
Sure the app could have been wittier. It could have been Al Pacino with “PaciNO or PaciYES”. It could have been even been political or perhaps somewhat useful in daily life, but Nic Cage is just a perfect target. After all, who else has an entire subreddit dedicated to praising him the way Nic does? His recognizable and rich media prowess is as inspiring as it is entertaining.
Additionally, Nic Cage is world renowned, so the conference talks from this endeavor would at least be an icon everyone knew. The playfulness even helped me come up with fun and interesting angles on why I’m searching for Mr. Cage. It didn’t take long before I found myself buying the domain DeclarationOfIndependenceThief.com for the talk.
With a few chats with fellow nerdy friends, the buzz was there. I knew it was time to get started.
How did it go?
Collecting training data went surprisingly well. There are a lot of great existing libraries out there for accumulating and mutating data. I went with Python for my resource collection and shimmed some simple structure on top of that with basic shell scripts.
The results of the data collection were entertaining. I’d be lying if I said I didn’t morbidly enjoy this process. I even enjoyed naming files.
After data collection: I had to leave my comfort zone
To take full advantage of the announced Apple ML training and tools, I installed the Apple beta software on everything. My phone was beta (iOS12), my desktop was beta, my Xcode was beta… my desktop was a walking experiment. And then I tried and failed — hard.
There was no shortage of issues. Betas failing, knowledge failing, resources failing, and in the end, I’m glad they did. I learned so much struggling through all of it, but I also got something beyond knowledge. I got new friends with similar interests. That was the real trick, the cool people.
I want to stop for a second and give kudos to the folks in the AI channel at Infinite Red. Kudos to Frank von Hoven for helping with the app, and especially thanks to Ray Deck who helped with CoreML and Swift.
After some exploring, lots of stumbling, and after my girlfriend thought I was completely crazy as my screen consistently flashed strange photos of Nic Cage, there was small progress.
In my talk video at the bottom, I explain a little more of the process, but creating the model took time.
At one point this complicated task finally clicked into place. Here’s the photo I took of my buddy Frank where the app properly identified face0
as “nic” and face1
as “not”.
Success! I started prepping my talk, bought a Nic Cage mask on amazon, and started trying to wrap my head around how I can release a beta app to the store with a 230MB Model dependency!
For my talk, I skipped the app store and just went with what you see in the video, but of course I’m not going to skip actually shipping such a useful (ok fun and novel concept) app!
Though the 230MB model was very accurate, I learned how to quantize model weights to create a “pretty accurate” 32MB version. See all the cool stuff I’m learning when I decided to ship?
Brush on some styles, a logo, a cool identifier box, and some settings 💥 you’ve got the full app ready for the app store. TADAAAAA 🎉🎉🎉🎉🎉🎉
Try the app! Let me know if you think of any cool features I should add. I even added settings for you to point at your own trained CoreML models in the cloud.
For now the app only works on iOS, but hopefully soon we’ll have it on Android, and even web.
I had to fight to get the app accepted into the App Store. They said “this isn’t something people can find useful” but we all know the real reason why it was rejected so many times…
How difficult is the code?
Thanks to the amazing advancements by Apple in iOS 12 and the awesome work by Ray Deck in wrapping those new libs in JavaScript, the code is quite clean and simple. I wrote a good bit of Swift at first, but Nic-willing that code will never see the light of day.
Instead, the final product utilizes awesome features in the react-native-vision module. Ray and I found a good medium where I could contribute constructively to the module.
Here’s the code to everything: sites, python, models, and app. Feel free to open a ticket if you find a bug, or PR if you think of a cool feature!
Presenting this madness was the fun part. I really enjoyed the smiles, giggles, and nods from my audience as I rambled on and on with code and screens like this:
If you’re interested in the actual talk, code, Nic cameo, and more the video is available here. I’d love to talk for 45 to an hour next time I give the talk. Need an entertaining keynote for your conf? Let’s chat.
The obvious reaction to such tech:
Yup, it’s that “Big Brother” scary technology that had throngs angered at Amazon when they allowed police to use it. Except for one thing, the Amazon service requires a round trip to their servers on an Internet connection, but Nic or Not is a proof of concept that operates solely on your phone.
“Aren’t you afraid tech like this will get in the wrong hands?”
The truth is, it’s already in the wrong hands. Fortune 500 companies have hardware dedicated to accelerated Machine Learning that can alter or even erase entire job markets, and I had to learn it existed from a TV show. It’s a cardinal time for every developer to become a player in the AI market, because only then can we create a friendly balance of power. We owe a lot to researchers, tinkerers, and students as our public connection to this awesome tech before it goes behind closed corporate doors. We need more of these champions. We need you.
Moral of the story
Machine Learning isn’t easy or simple, yet. Even the people I know who are great at it, claim to know nothing about it. So my advice to you is keep your ear to the ground. If you can, get started now, get excited yesterday, and fail your way into something new and beautiful. You don’t need to be an expert to have influence. There are already ML explorers, but not cultivators. Seed your thoughts with cool new tech, and let’s chat on Twitter.
Or if you just happen to think of me 😆 that’s cool, too.
If you want creative, collaborative, and friendly inventors to help you with your project, give us a hello@infinite.red — we also do onsite and online training, and yes it’s just as fun and crazy.
Interested in learning how to bring AI and Machine Learning to web with TensorFlow.js?
JavaScript lets you create front-end websites that can leverage the power of AI directly on the browser. Learn from scratch with this book.
Reserve your copy on Amazon now
Also: Special thanks to Chris who helped proofreadeded this post.
Now Join our ML and AI Newsletter: