New AI app for describing images and video: PiccyBot

This App is great for having videos described.

Hello Martijn!
I don't know if it would be possible, but would it be possible to describe what is happening at the exact moment when processing the video, like an audio description? Almost like seeing ai, but without the pause to describe.
Of course, it would have to give less information, but I think it would be amazing if it could do that. Reading the text or listening while the video is playing like this ends up causing synchronization.

Hello Martijn!
I noticed that the 2.14 version updated today mentions the ability to describe YouTube videos. This is truly a great upgrade. However, I haven't found a way to use this feature. I can't find PiccyBot in the sharing menu of YouTube videos, and copying and pasting the YouTube video link into the APP doesn't work either. Can you explain how to use this feature?
Many thanks

Hello Martin,
I discovered in the 2.14 what's new that some shortcuts were also introduced. This is a great joy. But please give us some details, what phrases they are exactly.

Hello Martin,
Thanks much for the info. I tried out all the shortcuts you had mentioned. Siri didn't reject any of them, but opened the main interface of Piccybot in reply to all of them instead of the camera or video recorder part. I note that I run an iOS older than 18 on purpose.
Thanks much for the language selection fix in 2.14. Now my native tongue, Hungarian can be found and selected in the language list, and not just by setting language to system language.
While browsing on Huggingface I found the following very promising uncensored model:
https://huggingface.co/huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated
This is the uncensored version of a very recent Qwen 2.5 vision model (the base model is developed by AliBaba group). This is the 7-billion-parameter variant, so computationally it is on par with the Janus Pro model that you currently run on your server. According to descriptions this is a quite versatile and strong model even in this size and multilingual too. It can process images in their native resolution and can be finetuned if needed. It can even process videos (even long ones) if the inference is done accordingly.
Personally I found nothing spectacular or special about the Janus Pro model, and this one is uncensored, so if feasible I propose to run this one besides, or even instead of Janus Pro on your server. Thanks in advance for considering this.

Hello Martin,
Thank you very much for your answer. I tried it just now and successfully selected PiccyBot from the share menu on YouTube. However, it's strange that when I posted a question to you a couple of days ago, I used the same method but it didn't work. Anyway, it's working now and it's functioning well.

deepseek r1 is wonderful, but my first impression of janus a while ago was oh well. it seems to have improved very significantly, now it gives a comprehensive description of the photo. Martijn must has done some good work about it.

Hello guys!
I don't know if I can report bugs in the Android version in this thread. I'm doing this since the developer looks here a lot, so it's easier for us to get support.
If necessary, I'll send it somewhere else.
I'm facing a bug where the description audio isn't being played.

I've been using this App for a couple months now and like it, but have noticed that it very often fails to process Youtube videos, either giving me the generic, "server error," message or just failing to process but then a retry will sometimes work, sometimes not. I do pay for the subscription. I have tried with multiple videos and multiple models. I tried running one of the same links through Gemini Flash to see if it was an issue with the model, but it gave me a very good description of the video broken up into time stamped segments, so the requests must not be going through the models directly which seems to make the App unreliable at the very least. I've read through all the comments here and haven't seen much about this, so either I have the misfortune of trying at bad times or it just isn't a widespread issue.

Hi Martijn,
Thank you for continuously adding new features to the app. As you mentioned in the post above, the app now has many features, especially providing many models for users to choose from. Could you please explain the advantages and disadvantages of each large model when describing pictures and videos? Of course, I know this question sounds a bit subjective, and perhaps everyone's opinion will be different, but I would like to hear your opinion, and I think there will be other friends who, like me, would like to have a reference answer.

Can you make the app guide the user so that they can take pictures with it?

Do you mean like Google Pixel's "Guided Frame"? That, would be sweet! 😃👍

I've been wanting something like that for ever now. But I guess you'd need live AI, I mean truly live AI for that?

Hello Martijn,
I found another bug in an app. I usually use Simplified Chinese, and I noticed that no matter which voice I choose, I often encounter situations where the content of the image description cannot be read out. The specific manifestation is that they will keep saying "Chinese Letter Chinese Letter Chinese Letter" until I have to pause the voice for it to stop. This is really frustrating because it happens frequently, about once for every five images I recognize this bug occurs. Could you find some time to check what's going on with this?

I noticed that too using the Chinese languages. Quite often the description gets stuck and starts repeating itself like a broken CD LOL

@Martijn if you replicate Guided frames or in other words make an accessible camera app for IOS, you would be doing a path-breaking, pioneering service for IOS users with visual impairment, not to mention facilitating their inclusion in mainstreme in a certain huge way. It need not be a feature within piccibot, rather it can be a stand-alone app, which can help us frame and take decent pictures and save to galary. I would be more than willing to contribute to it in any way I can.

I think Llama 4 has the fastest processing speed among these models, but its description effect is not as detailed as Gemini Pro2_5. However, considering both processing speed and description quality, Llama 4 is also a good choice. In my ranking of these models, it's Gemini Pro > Llama 4 > Deepseek.

Interesting addition, thanks. Completely understand why it's image only, video would both be more complicated, require more resources from the AI servers and you need to confirm how well it works before you even consider it.

Especially for situations where what you want is an accurate description and where halusinations can be problematic and/or dangerous.

I think, if you have truly found a way to remove these, you will become everybody’s new best friend.

This is something I've hoped would come along now when we have many good models that can be compared. Seems to work fine, but hard to say if it does not hallucinate of course. A possible further development: having a "fast" and a "thorough/detailed" mix, with modeels tailored to each?
Thank you very much for this!

Hi, I have downloaded the app but after using it for a while, I deleted it from my iPhone. I have Be My Eyes and Seeing AI so I feel like this app isn’t useful

No app will be useful to everyone depending on their situation and needs, that's perfectly valid. Just bear it in mind as a potential tool in the future if you need a short video described, or if you encounter a situation where the AI models in Seeing AI or Be My AI aren't helpful.

Hi there,

I know this might be a bit of an unusual request, but I wanted to put it out there anyway—because if you don’t ask, you’ll never know what’s possible.

Lately, I’ve been using a few survey websites to earn gift cards like Amazon gift or prepaid Visa cards. These help me make online purchases without needing to use my debit or credit card. One of the main platforms I use is powered by Spectrum Surveys, and while the surveys themselves are accessible most of the time, there’s one issue I keep running into: drag-and-drop tasks.

These questions usually ask me to move items—like dragging an image or word into a specific box or column—but there’s no reliable feedback about what I’m dragging or where I’m dropping it. This makes it nearly impossible to complete certain sections.

Here’s where the idea comes in. I was wondering if it would be possible to create a feature that allows me to share my screen and receive descriptive feedback about what’s happening. For example, if I’m dragging “Apple” into a “Fruit” box, the system could announce something like:
“Dragging Apple. Drop zone: Row 1, Column 1 – Fruit.” And once I release it: “Apple dropped in Fruit.” If I switch to dragging “Blue” into a colors section, it could say: “Dragging Blue. Drop zone: Row 1, Column 2 – Colors.” Then confirm: “Blue dropped in Colors.”

It doesn’t need to control anything for me—I’m not expecting the AI to do the action itself. I’d simply like to hear clear, accurate feedback so I know what I’m doing on screen. This could work in combination with screen recognition when standard reading fails, which is something I already use regularly.

I think having real-time feedback on selections and drop targets would significantly improve my experience and make these survey platforms far more accessible. This kind of assistance wouldn’t fix every challenge, but it would eliminate one of the biggest barriers I run into.

Also, I plan on purchasing the lifetime subscription soon—probably by Friday. I have a $25 prepaid Visa card ready, and I believe the lifetime cost is around $20. Regardless of whether this feature is possible now or in the future, I’m fully committed to supporting the app. I figured I’d share this idea in case it sparks anything or aligns with something already in development.

If this kind of feature is technically possible, it would be a game-changer for me. And if others feel the same way, hopefully they’ll chime in and add their thoughts as well.

Thanks for your time and for building something that’s already made such a difference. I’m looking forward to what comes next.

I like to keep some prompt histories so I don't have to retype them all the time, or go copy them and paste them in from elsewhere. I have noticed the latest version, the APP keeps randomly refreshing as if I closed it and re-opened it, and that keeps erasing my prompt history. Is it those random tips that's doing that?