Hello guys,
I have created the free app PiccyBot that speaks out the description of the photo/image you give it. And you can then ask detailed questions about it.
I have adjusted the app to make it as low vision friendly as I could, but I would love to receive feedback on how to improve it further!
The App Store link can be found here:
https://apps.apple.com/us/app/piccybot/id6476859317
I am really hoping it will be of use to some. I have earlier created the app 'Talking Goggles' which was well received by the low vision community, but PiccyBot is a lot more powerful and hopefully useful!
Thanks and best regards,
Martijn van der Spek
Comments
great app for video describing
This App is great for having videos described.
Audio description?
Hello Martijn!
I don't know if it would be possible, but would it be possible to describe what is happening at the exact moment when processing the video, like an audio description? Almost like seeing ai, but without the pause to describe.
Of course, it would have to give less information, but I think it would be amazing if it could do that. Reading the text or listening while the video is playing like this ends up causing synchronization.
How to describe a YouTube video?
Hello Martijn!
I noticed that the 2.14 version updated today mentions the ability to describe YouTube videos. This is truly a great upgrade. However, I haven't found a way to use this feature. I can't find PiccyBot in the sharing menu of YouTube videos, and copying and pasting the YouTube video link into the APP doesn't work either. Can you explain how to use this feature?
Many thanks
2.14: what are the available shortcuts?
Hello Martin,
I discovered in the 2.14 what's new that some shortcuts were also introduced. This is a great joy. But please give us some details, what phrases they are exactly.
Sharing YouTube and Shortcuts
Carter, PiccyBot will describe videos that are shared to it. In the case of YouTube, go to share, then select PiccyBot. The first time it will be hidden under 'more..' and then again 'more..' to find it. After that it will show up earlier.
However, there appears to be a glitch in the YouTube app at the moment, likely related to the iOS update, that somehow causes the share function not to work. For some people it has started to work again. So be aware of that please. Using YouTube in Safari works fine.
Laszio, PiccyBot now has shortcuts, but it is an initial release. Please check it out and let me know what to improve or add?
Right now, with the 2.14 update, Siri will recognize these phrases to trigger the camera shortcut:
"Siri, Open PiccyBot camera",
"Siri, Launch PiccyBot camera",
"Siri, Start PiccyBot camera"
Siri will recognize these phrases to trigger the video recorder shortcut:
"Siri, Open PiccyBot video recorder",
"Siri, Launch PiccyBot video recorder",
"Siri, Start PiccyBot video recorder"
However, again there seems to be a glitch, it doesn't work for everyone, likely related to the iOS 18 updates to enhance SIri. So I didn't announce this functionality yet, let's test it a bit further first.
Shortcuts + uncensored model proposition
Hello Martin,
Thanks much for the info. I tried out all the shortcuts you had mentioned. Siri didn't reject any of them, but opened the main interface of Piccybot in reply to all of them instead of the camera or video recorder part. I note that I run an iOS older than 18 on purpose.
Thanks much for the language selection fix in 2.14. Now my native tongue, Hungarian can be found and selected in the language list, and not just by setting language to system language.
While browsing on Huggingface I found the following very promising uncensored model:
https://huggingface.co/huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated
This is the uncensored version of a very recent Qwen 2.5 vision model (the base model is developed by AliBaba group). This is the 7-billion-parameter variant, so computationally it is on par with the Janus Pro model that you currently run on your server. According to descriptions this is a quite versatile and strong model even in this size and multilingual too. It can process images in their native resolution and can be finetuned if needed. It can even process videos (even long ones) if the inference is done accordingly.
Personally I found nothing spectacular or special about the Janus Pro model, and this one is uncensored, so if feasible I propose to run this one besides, or even instead of Janus Pro on your server. Thanks in advance for considering this.
My YouTube description was successful
Hello Martin,
Thank you very much for your answer. I tried it just now and successfully selected PiccyBot from the share menu on YouTube. However, it's strange that when I posted a question to you a couple of days ago, I used the same method but it didn't work. Anyway, it's working now and it's functioning well.
significant improvement on deepseek ai janus
deepseek r1 is wonderful, but my first impression of janus a while ago was oh well. it seems to have improved very significantly, now it gives a comprehensive description of the photo. Martijn must has done some good work about it.