SIMD-accelerated computer vision on a $2 microcontroller

(shraiwi.com)

120 points | by zdw 21 hours ago

9 comments

  • camel-cdr 2 minutes ago
    The CH32V407 is a bit more expensive but supports the embedded subset of the RISC-V vector extension.

    So you basically get a full AVX-512/SVE level SIMD ISA, minus FP support and with 64-bit SIMD width. But you still have LMUL, which will be great here to maximize utilization.

  • jwr 2 hours ago
    BTW, I find it pretty amazing that you can buy complete ESP32-S3-Zero boards on AliExpress for $5.33. That's for a little board with the chip, a ceramic antenna, a USB connector and an LED.

    ESP32-C3-Zero are $2.80 each.

    These prices are absolutely ridiculous for what you get. Also, the ease of use of these boards is incredible. Just connect them via USB to your computer and off you go.

    • twisteriffic 11 minutes ago
      Anecdotal, but believable, warning for those in the US

      https://www.reddit.com/r/GlobalEntry/comments/1x4c062/us_ord...

    • javawizard 45 minutes ago
      Wait until you get into developing your own PCBs and discover the BK7238 for $0.81 :)
    • htrp 1 hour ago
      what do you do with them?
      • KZerda 6 minutes ago
        Personally, I just finished up a project with one to poll outdoor and indoor weather sensors and notify the household over discord when it's warm enough outside to probably need air conditioning, when it's cooled off enough to probably not need AC, when the AC goes on, and when the AC goes off. It also parses and displays and notifies information from the National Weather Service when there are relevant alerts in my area.
      • ssl-3 13 minutes ago
        Cheap MCU dev kits in general, for me?

        A somewhat-elaborate fan speed controller for a wonderful old pro audio power amp that also needed a good bit of other rehab

        A window fan controller, for summer use, that only turns on the fan when it's cooler outside than it is inside -- so it only cools the indoors, seeks never to heat it

        A sensor for automating the color temperature of my indoor lighting based on outdoor conditions

        A macro pad that ultimately works kind of like a Streamdeck, but without the built-in display features (that was a download-and-print project)

        Other random stuff, like an ESPHome widget that talked to a window aircon via infrared commands (mostly just a download-and-run project)

        They're just cheap little programmable low[ish]-power computers on a board, with some flash storage and RAM, and a fair bit of low-level IO. Often with wireless network connectivity as a built-in. They do stuff. :)

        If the code is tight-enough, they can do a remarkable amount of that stuff -- in terms of raw compute, they present something near-ish to the Pentium era of personal computing for a few dollars.

        For me, they make sense for simple projects. When something like a disused laptop or a Raspberry Pi-like device [and the accoutrements, like a whole-ass operating system] seems like overkill, then: It's probably overkill.

        With a dev board, I can breadboard a circuit and maybe even leave it that way forever, or get a custom PCB drawn up with KiCad and made inexpensively by JLC PCB or Osh Park or whatever, or just solder wires and parts to the dev board like a caveman.

        All methods work, and the cost of screwing it up is usually very low.

        After that, the thing gets powered up with one of the featureless USB wall warts that we all have accumulated too many of and it just sits there and does its job.

        (And honestly, stuff like the Pi Pico and random ESP32 dev boards are also overkill for what I do with them. A lot of the stuff I've done could be accomplished with an MCU chip that costs pennies instead of dollars, but my quantities are low-enough that I just don't care much about a few dollars for occasional one-off projects.)

      • jwr 53 minutes ago
        I am currently working on a Bluetooth Mesh home network allowing me to control lights, gather sensor data from various places, etc (think interfacing with a home alarm system for presence detection).

        Also, a home air quality monitor, both with a display and reporting to my mesh network.

        Or you could build a "plane radar" like this one :-) https://github.com/ironicbadger/ESP32-Plane-Radar

      • teaearlgraycold 21 minutes ago
        Similar class of chip, but I use the rp2350 to perform computer vision with LiDAR to detect when people leave dishes in a sink and beep angrily at them.
  • pjmlp 4 hours ago
    > For its price, the ESP32-S3 is a powerhouse of a microcontroller.

    Indeed, don't forget it is more powerful than computers like this one,

    https://en.wikipedia.org/wiki/Amstrad_PC1512

    • asimeqi 3 hours ago
      That was my exact first computer with the CGA graphics, GEM Desktop, Locomotive Basic etc. All the software came in 4 360KB diskettes. The diskettes had each it's own beautiful color. There were 3 sample Basic programs that seemed to me quite advanced. One sample was a document management system. Another was a drawing program where you could draw with a mouse. Didn't know that GEM Desktop was made by Digital Reasearch. It was a very good windowing system. I believe it was based on MS-DOS. There was also a CP/M OS. So much software in only 1.2MB.
      • sedatk 1 hour ago
        I dearly remember the keyboard's warm red Caps Lock light and my sessions of running Print Master and writing BASIC programs on it.
      • pjmlp 2 hours ago
        Exactly, thanks for sharing.
    • MeteorMarc 2 hours ago
      We will see the day your toothbrush has more compute than the original IBM PC.
    • glouwbug 2 hours ago
      At 8mhz? I'd imagine it rivals at least the Pentium III
      • pavlov 2 hours ago
        Well, probably not, but maybe a plain Pentium?

        The Pentium III was a superscalar core that scaled up to 1.4 GHz. This microcontroller is max 240 MHz and does less work per cycle.

      • pjmlp 2 hours ago
        Probably, I was use this example because I feel youngers can't imagine how much we could do with such MS-DOS class computer, even within its limitations.

        So naturally an ESP32 can do so much more.

  • lukan 55 minutes ago
    The engineering seems impressive, but how useful is the result? What can you detect exactly with it in real time?

    Detecting corners, but to what resolution/detail, a picture with a result would have been helpful to understand the capability.

    edit: missed this link, this is what this is about

    https://en.wikipedia.org/wiki/Features_from_accelerated_segm...

  • peesem 3 hours ago
    nit: 5.1MP/s to 11.2MP/s is not an increase of 220%, it's an increase of 120%. it's 2.2x faster, but percentage increase starts at 0%, not 100%. 100% faster is double, 200% faster is triple.
    • seritools 3 hours ago
      nit²: 2.2x as fast, or 2.2 times the speed; not 2.2x faster. same issue you pointed out.

      using faster with x/times is just begging for misunderstandings

    • HPsquared 1 hour ago
      2.2x the speed is 1.2x faster.
  • paaloeye 3 hours ago
    ESP32-S3 SIMDs are super difficult to use unfortunately
    • PowerElectronix 2 hours ago
      Practice makes perfection
      • paaloeye 2 hours ago
        true, but nothing is perfect
    • dsign 1 hour ago
      I find it easier than x86 SIMD[^1], but the lack of support for vector operations for 32 bits makes me very grumpy.

      [^1] Though, to be honest, x86 has great automatic SIMD tools so I never had to try too hard.

    • meindnoch 3 hours ago
      Just LLM it
      • ballmerpoint 3 hours ago
        IME, they’re still pretty bad at getting specific implementation details of SIMD correct. The best I’ve done so far is to harness them to an actual piece of hardware and have them iterate until they see performance improvements but 1) Goodhart’s law, they tend to overfit the benchmark suite and 2) the resulting SIMD often becomes illegible.
        • TGower 2 hours ago
          For a lot of SIMD applications, overfitting isn't really a concern. In my use case I need to calculate the sum of absolute difference between two vectors for autocorrelation, there's no real possibility for overfitting, either it gets the correct answer or it doesn't. LLMs are good at reading through the CPU manual and trying different optimizations.

          The resulting SIMD being illegible isn't an issue in my eyes, assembly isn't a language known for it's readability in general. If you want to really understand what the assembly is doing, ask your LLM to whip up a visualization and explanation tailored to your input data where you can see how each operation leads you to the final result.

        • pezgordo 1 hour ago
          Have you tried including the instructions documentation to the project?
          • ballmerpoint 14 minutes ago
            Yes, of course. It doesn’t seem to recall the specific details very well, or confuses them with x86 or ARM instructions that do similar things. And yes, my AGENTS.md says ONLY USE INSTRUCTIONS IN blah.txt.

            I dunno, clearly I need an LLM psychiatrist.

      • paaloeye 2 hours ago
  • paaloeye 2 hours ago
  • adolph 45 minutes ago
    Interesting. Since this was implemented on the S3, which uses the ISA XTensa LX instead of the RISC-V of most newer ESP32 series chips, I looked into how Espressif is handling SIMD on the latter ISA. Apparently it is only present in limited fashion as a processor extension on the P4 and S31 [0]. The S31 datasheet claims SIMD [1] and the P4 datasheet [2] files them as "Processor Instruction Extensions."

    0. https://bitbanksoftware.blogspot.com/2026/04/esp32-p4-simd-e...

    1. https://documentation.espressif.com/esp32-s31_datasheet_en.p...

    2. https://documentation.espressif.com/esp32-p4_technical_refer...

  • varispeed 1 hour ago
    Why is it still called SIMD-accelerated, if these are now just normal instructions. You just have to take a little bit more effort into restructuring your algorithms to make use of them.
    • anthomtb 1 minute ago
      How does using instructions that operate on multiple data elements, which make your software 3 times faster, not fit the definition of SIMD-accelerated?