The $70 Raspberry Pi AI Camera comes only a few months after the identically-priced Raspberry Pi AI Kit, the company’s first foray into AI accessories, and promises an even more versatile way to get started with various AI projects.
Raspberry Pi sure is doubling down on artificial intelligence with its recent accessory releases. The excellent Raspberry Pi AI Kit brought 13 TOPS of AI compute to the Raspberry Pi 5 by strapping a general-purpose Hailo-8L M.2 AI accelerator module onto it using the power of the PCIe-based M.2 HAT+.
The new Raspberry Pi AI Camera takes a different approach, bundling the AI accelerator together with the image sensor itself and exposing all the AI goodness through the camera connector, a standard interface that’s been present on almost every Raspberry Pi board for years now. This is all facilitated by the new Sony IMX500 image sensor (more on it in a second), with Raspberry Pi’s own RP2040 making a surprise cameo handling comms duty.
If you haven’t already, consider checking out our in-depth review of the Raspberry Pi AI Kit, as there will be comparisons between the two in this article – especially given how different yet similar they are in some regards. While we’ll stop to explain and provide some context where needed, having some prior insight into how the Raspberry Pi’s AI ecosystem works – and why you’d even want local AI in the first place – can help tremendously.
We want to thank Raspberry Pi for providing us with a review unit of the camera. It’s a pre-release copy which came in generic packaging, so we can’t take photos of the very snazzy retail box or say much about the unboxing experience. And for those playing along at home: if you see any hardware differences between your unit and the one in our photos, don’t worry too much about it – this is the reason why.
Hardware overview
The physical design of the Raspberry Pi AI Camera will come as no surprise to anyone who’s ever handled any of Raspberry Pi’s other camera modules. With its 24 x 25 mm footprint and all the mounting holes in their standard locations, the Raspberry Pi AI Camera should be a drop-in replacement (as long as you make sure that there’s extra clearance in your design for the bulkier sensor package and extra components on the PCB itself). The little 3D printed camera holder we’ve been using for a while now still works just fine.
Sony’s IMX500 Intelligent Vision Sensor is behind all the AI magic here. By combining a 12.3 MP image sensor and an NPU into one single package, it enables efficient on-device inferencing without the need for any extra AI acceleration hardware. Sony unfortunately doesn’t provide a TOPS rating (and is in general a bit secretive about the sensor), but does list a Full/video + AI processing framerate spec on their key specifications page. The associated 30 FPS figure is a bit open-ended as there’s no info on the AI model used to obtain it – or is it some sort of hard limit? Does the video framerate automatically get capped to 30 FPS when the built-in NPU comes online to prevent overheating? We can’t know for sure unless Sony decides to push out the complete datasheet.
The sensor’s NPU doesn’t operate on the full-resolution video stream, but rather a scaled-down input tensor produced by a simple on-board ISP. The largest input tensor which can be handled has a dimension of 640 x 480 pixels (though naturally, more demanding AI models will likely require even lower input resolution, which can be dialed all the way down to just 64 x 48 pixels). In turn, the NPU spews out an output tensor which gets sent over to the host system to be displayed or for any extra computation required.
Unlike regular camera modules which only stream image data, the Raspberry Pi AI Camera thus has an extra inference stream to push out. This stream, containing all the output tensors, gets sent to the Raspberry Pi over its CSI-2 camera interface – the same one already used by the image stream. Bunching the two up together is a very sleek solution for a number of reasons, but most importantly it allows the Raspberry Pi AI Camera to work on every single Raspberry Pi system equipped with a camera connector.
Some of you might recall that we mentioned an RP2040 a few paragraphs ago. What’s it doing in here? Well, this chip mainly facilitates file transfers and acts as an intermediary between the OS and the IMX500, functioning as an I2C-to-SPI bridge. In order to run an AI model, the IMX500 has to load it into its internal 8 MB memory. Naturally, models have to first be quantized, compressed, and packaged into something Sony calls a firmware file that’s compatible with the sensor’s NPU – and care needs to be taken to ensure that these actually fit into the very limited space available (especially since the 8 MB is shared between model storage and runtime memory).
Hold up – we hear you say – just 8 MB of memory? Why not just share the Raspberry Pi’s significantly larger RAM?
There are a few reasons. The IMX500 is designed around the concept of being fully self-contained, memory included. AI models require fast memory access, and the only way to facilitate that when sharing system memory is by using a suitably fast bus for connecting your AI gizmo to the host hardware. PCIe is a logical choice here, which is precisely why many standalone AI accelerators come as M.2 modules. This works great as evidenced by the Raspberry Pi AI Kit, but does come with a caveat: the only Raspberry Pi board which features a PCIe connector is the Raspberry Pi 5! By having its own memory pool, the Raspberry Pi AI Camera has to send much less data back and forth and can make do with just the camera connector, making it compatible with all Raspberry Pi boards.
Okay, that explains the benefit of having memory integrated right into the camera, but still doesn’t explain why there’s so little of it. We have no way of benchmarking how fast the chip’s on-board memory is, but we have reasons to suspect it’s faster than conventional RAM, likely being similar in design to CPU cache memory. This type of memory is significantly more expensive, which could explain the 8 MB figure – but ultimately, it’s likely that the choice boils down to Sony simply having to cost-optimize the IMX500.
The IMX500’s memory is volatile, so Raspberry Pi included an extra 16 MB of on-board flash storage specifically meant for caching recently used firmware files. This should allow for much quicker model initialization as opposed to beaming the data via I2C every single time. That built-in RP2040 handles all the caching automatically – so you likely won’t even know that it’s happening. Neat!
Let’s turn our attention to the camera part of the entire “AI Camera” concept – and for what it’s worth, it’s a pretty capable little imaging setup. The IMX500 is based around a 12.3-megapixel (4056×3040) 1/2.3 type sensor with a pretty decent readout rate – up to 60 FPS when shooting full-resolution and 4K video and 240 FPS for 1080p video as per Sony’s spec sheet. With that being said, the Raspberry Pi’s own encoding speed limitations cap camera framerates to 60 FPS for 1080p video, and ~10-15 FPS for 4K footage (mind you, these numbers are for the Raspberry Pi 5, and older models could perform worse). And then you’ve got the camera interface bandwidth which in practice limits framerates even further, usually down to 10 FPS for higher-resolution streams, and somewhere between 30 FPS and 50 FPS for 1080p video, depending on the sensor.
Raspberry Pi’s IMX500 driver implementation offers only two video modes: 4056x3040p10 and 2028x1520p30. We’re not sure whether additional modes were omitted given the camera’s AI-centric intended use (because of that mysterious 30 FPS Full/video + AI processing limit), or whether there was simply no suitable video mode on the sensor itself that could provide a faster framerate without overloading the Raspberry Pi in some way.
The AI Camera has no autofocus, but the small sensor size gives them a pretty deep depth-of-field that stretches from 20cm all the way to infinity. If need be, focus can be manually tweaked by twisting the lens.
This is all to say that the Raspberry Pi AI Camera can be used as a decent stills and video camera in a pinch – but if that’s all you’re after, the $70 asking price is a bit much. The $25 autofocus-enabled Raspberry Pi Camera Module 3 is a much, much better choice, especially when you take into account its wider range of video modes and even some slow-mo (120 FPS) shooting options.
But instead of talking about what the new camera isn’t meant for, let’s focus on its AI features. In order to check them out we have to turn to the software, which leads us to…
Hands-on with the Raspberry Pi AI Camera
Raspberry Pi’s hardware is excellent, but it’s the software support that sets it apart. Just like the Raspberry Pi AI Kit, the AI Camera integrates tightly (perhaps even more so) with Raspberry Pi’s existing software stack. The easiest way to get started is using rpicam-apps, Raspberry Pi’s handy camera application suite, but full integration with the Picamera2 Python library is also available for those needing programmatic access to the IMX500 from their Python scripts.
But before you can run anything or start tinkering away, it’s necessary to download the imx500-all support package using apt which contains all the required firmware files, post-processing stages, AI models to get you started and Sony’s model packaging tools.
Just like with the Raspberry Pi AI Kit, the AI Camera’s integration magic with rpicam-apps happens through post-processing stages. It’s best to imagine these as filters of sorts which occupy slots in the post-processing pipeline. Each stage performs a certain operation (or set of operations) to the image – be it simple things like color filtering or superimposing text, or more involved computing tasks like AI operations.
There are two built-in demos. These somewhat mirror the ones we’ve already seen running on the AI Kit’s Hailo-8L. In fact, while the Hailo-8L demos use different AI models, it seems that the same respective visualization stages got reused for object detection and pose estimation demos on both platforms. Let’s first run the MobileNet SSD-based object detection demo:
rpicam-hello -t 0s –post-processing-file /usr/share/rpi-camera-assets/imx500_mobilenet_ssd.json –viewfinder-width 1920 –viewfinder-height 1080 –framerate 30
Woah – pretty sleek! It’s kind of impressive that most of what’s happening here is going on in the little camera module, with the host Raspberry Pi doing minimal processing work to simply render the scene and object detection boxes in the preview window. This is also evident in Raspberry Pi OS’ task manager – the CPU is mostly idle, with its utilization sitting at ~10%. Awesome.
Now onto the second demo, a pose estimator based on PoseNet:
rpicam-hello -t 0s –post-process-file /usr/share/rpi-camera-assets/imx500_posenet.json –viewfinder-width 1920 –viewfinder-height 1080 –framerate 30
Works just as well – and the CPU load stayed similarly low. On the Raspberry Pi AI Camera, both of the demos we’ve run work with an input tensor size of 640 x 480 pixels. There were no framerate drops, with the NPU delivering a smooth 30 FPS, and the whole experience was genuinely good. From an end-user standpoint, we couldn’t be happier.
Aside from freeing up the CPU for other tasks, AI accelerators generally improve a system’s power efficiency. We were measuring power draw of the entire system during the demos, and got a reading of 6.5 W at the wall. If we assume a ~90% power efficiency figure for the PSU, that works out to just 5.85 W. Impressively low!
In fact, this means that this IMX500-based setup draws some 40% less power than the already extremely efficient Hailo-8L setup from the Raspberry Pi AI Kit. The latter draws around 9.7 W during inferencing – though it provides significantly more performance.
To put both of these numbers into perspective, running AI tasks on the main CPU puts quite a hefty strain on it, which also shows up in the power draw. We managed to obtain a ~13.3 W full-system figure during testing – and the performance wasn’t great at all.
When loading a model up for the first time, there’s nothing cached in the on-board flash, so the entire firmware file has to get copied over from your Raspberry Pi onto the AI Camera. This can take a little while. Once cached, the firmware loads within seconds during subsequent runs. We’re also pretty happy to report that the system is capable of keeping not just one, but several recent firmware files in cache.
That’s great – but what about developing your own applications or even bringing custom AI models? If you want to stay in the rpicam-apps environment, Raspberry Pi provides a very handy IMX500PostProcessingStage base class which you can derive custom post-processing stages from where you get very granular control over what happens to your output tensors once they make it over to your Raspberry Pi.
If you’re looking to develop a completely custom app you should look into the Picamera2 library. The integration works in much the same way it does with rpicam-apps. There’s an IMX500 helper class which provides all the utilities required for making calls to the sensor from a Python script. There’s a whole lot you can do with it, and Raspberry Pi’s excellent documentation covers it in detail – and there’s even a little demo repository with three Python examples (object classification, detection and segmentation) and a Model Zoo with a whole bunch of ready-made models to be used with these.
But we haven’t yet answered the big question we posed a second ago: what about custom models? In order to bring your own AI model, it’s necessary to pack it into a firmware file. Sony has a set of free tools meant to allow for streamlined model quantization, compression and packaging, allowing you to convert most any PyTorch or TensorFlow model into a compatible format (as long as it can fit the limited memory available on the IMX500). Once uploaded to the Raspberry Pi AI Camera, your custom model is ready to use!
Verdict: AI Camera or AI Kit?
Raspberry Pi currently offers two very different AI solutions. Sure, the software stack used by both is now mostly unified, and the capabilities of the two systems seem broadly similar. With both boasting the same price, how does one choose?
Well, let’s start with the most obvious difference. The AI Camera is a lot less powerful than the AI Kit. With its 13 TOPS of AI compute and PCIe access to gigabytes of system memory, the Hailo-8L is a beefy chip. Come to think of it, it’s probably not too much of a surprise to anyone that a dedicated NPU outperforms one embedded into a sensor – but that’s beside the point.
Quantifying how much faster it is, however, is somewhat problematic. Furthermore, the IMX500’s true Achilles heel is its memory. With just 8 MB at its disposal for pretty much everything AI-related, it’s a real limiting factor. And while the Hailo-8L won’t be running any LLMs either, it certainly takes a lot more to bog it down compared to the IMX500. For crying out loud, some of the heftier models from the Model Zoo run at ~20 FPS at rather low resolutions (256 x 256 and lower).
The AI Kit also offers an (unofficial, for now) upgrade path which we’ve explored a little in our review of it: it’s possible to swap the Hailo-8L module with the Hailo-8, doubling the inferencing performance to 26 TOPS.
This isn’t to say that the IMX500 isn’t a capable little AI accelerator in its own right, as it’s proved its worth in the rpicam-apps demos we’ve ran. For most projects, it’ll be more than enough and it’s a marked improvement compared to running AI models on the Raspberry Pi’s CPU and actually makes real-time inferencing more than possible in a lot of projects.
The AI Camera works on pretty much every Raspberry Pi board you might have, while the Raspberry Pi AI Kit requires a Raspberry Pi 5. And the AI Kit simply packs a lot more bulk, takes up the PCIe connector, requires the use of an adapter HAT+ and a separate camera module on top of that. Over on the AI Camera side, it’s perfectly possible to pair it up with a Raspberry Pi Zero 2 W and end up with a capable setup that’s absolutely minuscule.
It might just be us, but we found the AI Camera a bit easier to use than the AI Kit. Take this with a grain of salt, as both of these have excellent software support and docs, but the former seemed just a tad better integrated with the software, and taking further steps after being done with the demos was just a little simpler.
And finally – price. While both of these systems have the same upfront cost – $70 for both the AI Kit and the AI Camera – unless you’ve got one already, the AI Kit does require you to get a separate camera module. You also have to budget for a Raspberry Pi 5 if you’re still using an older board. This can add up to over $150 if you’re buying all three together. Conversely, even if you’re still using something like a Raspberry Pi 3B, the AI Camera will always set you back just $70.
You can’t really go wrong with either option, but we suspect that most tinkerers and casual users will be more drawn towards the latter. It’s certainly what we’d recommend. Simply grab whatever Raspberry Pi board you’ve got lying around, slap the Raspberry Pi AI Camera on and you’re ready to go. It’s the most sensible way to dip your toes into the world of AI.
- Raspberry Pi AI Camera review: Even more approachable AI - 10/22/2024
- Arturia AstroLab review - 09/29/2024
- Raspberry Pi AI Kit review - 08/16/2024