Copilot+ PCs are the first computers to run Small Language Models (SLM) on-device. The benefit of this approach is that it produces results for tasks like image generation or text generation quicker than the cloud-based Copilot app. Now, Microsoft has introduced AI Dev Gallery, an easy way to integrate on-device AI functionalities in any app.
The AI Dev Gallery app is for developers who want to try multiple models to integrate an AI feature into their app. The app offers over 25 samples you can download and run on your device. Moreover, you can export the project or source code directly into your app and get it working instantly. It works on Windows 10 and 11, on both x64 and ARM64 architectures.
Out of curiosity, Windows Latest cloned the AI Dev Gallery from its GitHub repository. Currently, the only way to access it is by building the project in Visual Studio and then running it. In addition, you'll need at least 20GB of space and a multi-core CPU. It also suggests a GPU with 8GB VRAM, but those are for heavy models only.
We started with a 4-core CPU, 4GB RAM, or a Windows 11 PC. The app has two modes: Sample and models. We used the sample view to check out all the available models. The app categorizes them into Text, Image, Code, Audio and video, and Smart Controls.
Models for image generation and video generation are quite big, nearing about 5GB. We started with a small one related to upscaling, which was under 100 MB. We took a screenshot of the screen and tried to upscale it using the CPU. Note that you can switch between CPU and GPU for processing the request.
The upscaling happened in under 30 seconds in this inferior virtual machine, where the RAM consumption climbed to 1GB momentarily. The app showcases an upscaled version of the image with a 9272*4900 resolution. The image elements, especially text, suffered a severe hit and were illegible.
There's no option to preview this generated image in a bigger window or full-screen. Even the download option isn't there to save it to disk.
We tried another model called Detect Human Pose. It identifies someone's position in an image. While it accurately identified a basic walking person, it even started showing position marks for screenshots of our desktop with a few apps open.
We don't know how these models can be integrated into the app, but some of these features can be run locally. Of course, PCs will need bigger storage for these models and powerful CPUs with 16GB or more RAM.
What do you think? It is worth downloading a 5GB model to convert a text prompt into an image or wait for 30 seconds to do that on a web app. Clearly, most of these features have very niche use cases and implementation environments rather than appealing to the whole Windows 11 user base.