A ChatGPT Powered Voice-Command Online Fashion Webshop for the Visually Impaired Community

By Juies, 9 February, 2024

Forum
App Development and Programming

Hello r/Blind community,

I've been working on an accessible online fashion webshop for the visually impaired, using OpenAIs models for intuitive voice command navigation and detailed audio descriptions. It's designed to offer a seamless shopping experience, but it's still a prototype, meaning you can't make purchases yet.

Quick Tutorial:

- Navigation: You can submit commands by simply holding the spacebar, saying what you want into the microphone, and letting go of the spacebar. These commands can be anything: Filter for products, go to a certain product page, add a product to cart, ask for an explanation of the website, navigate to the home page, etc. The voice commands don't have to abide by certain standards, just say what you want and the system will figure out what functions to call.

- Searching: Simply describe what you're looking for, like "find blue jeans."

- Go to a specific product page after searching: Ask for product descriptions. Each product will have a number that will be called out after you searched for the type of product you want. Just call out this product number and it will navigate to the proper page, like: "Show me product 8 in more detail".

Link: www.planetdurable.nl/test.html

Just click the link and you're ready to search for some products by holding down the spacebar making a voice command.

Just wanted to share. Let me know if you guys have any comments on it.

Kind regards, Julian

Options

Comments

By Ekaj on Thursday, February 8, 2024 - 21:44

I just tested this out and it seems to work great on my Mac. I only gave a few commands, but the voice you are using is very natural and clear.

By Brad on Thursday, February 8, 2024 - 21:44

I'm trying it and at the moment I have the following to note:

1. It says I don't understood instead of I don't understand.

2. Perhaps a sound could play indicating the command was understood.

3. I tried searching for trousers extra large and it gave me t-shirts. I'm assuming trousers isn't a word you use?

The reason I say I don't understand why this is needed is because screen readers can do all this and in most cases we'd probably be faster.

It's a nice idea but I don't think you've fully looked into screen readers and what they can do.

This might help for someone who is older though, I'm not sure.

Also the chat gpt costs would be quite overwelming and I doubt anyone would pay for this.

I'm trying to think if I can think of a time I'd use it and honestly, I can't. Not even if I was feeling lazy. The real issue with speach stuff like this and Smart speakers is you can't skip what's being said like you can with a screen reader, you have to listen to everything.

By Ash Rein on Thursday, February 8, 2024 - 21:44

Is there a reason why it would not e useful? And to assume that this person does not know enough about screen readers is worrying. They are trying to help us. It provides us with another avenue to find useful products we might buy.

I am definitely interested in this and so far, it has worked relatively well. Something that I would find useful is if it provided what fabric type, the weight, etc. This is because it can be very difficult to find information like that. The world is transitioning to AI and more directly asking for what we want. This can be a good step towards that.

The website is definitely built from someone living in Europe. So there could be a language barrier. But that in itself, is it necessarily an issue right now. As it gains popularity and utility, it will be easy enough to approve the language challenges. I think adding things like more specific colors can be a useful thing as well.

In truth, anyone coming on this website, looking for feedback, deserves good feedback. And, positive feedback. If only because it encourages them to continue doing it and come up with other ideas that can be useful to blind people.

By Siobhan on Thursday, February 8, 2024 - 21:44

Whether I or anyone would use this app doesn't matter in the slightest. Whether it says I understand or understood, doesn't matter. What matters is vivid descriptions of actual colors. None of this sage, or cool blue junk, how about green, red, brown. Perhaps it can suggest things like a red tank top will go great with those blue knee-length shorts. Visual things matter to some of us so I'd like to know the outfit either I pick or AI picks out, is in fact matching.

By Brad on Thursday, February 8, 2024 - 21:44

In order for the product to be fully understood, correct English must be used.

It's in beta at the moment so it's not an issue but that would be one of the first things I reported if it wasn't.

I don't think the AI can do that at the moment because you don't have any input for it but if you uploaded a picture it might be able to help you.

@ash rein, call me old school then, perhaps people will use this, I just prefer browsing the older way.

Also I think turms like, jumper, trousers,, trainers, beany, and more that I can't think of should be added for those who use UK English.

By Siobhan on Thursday, February 8, 2024 - 21:44

I'd rather get the main accessibility functions, descriptions of colors if applicable, then worry about what something's called, or if it is misunderstand or whatever it is. If english is so important, then watch how generation A and Z write now, without using ' and other decent english language concepts. I don't judge, just accept and figure things out. I see no sense in making an app with UK and American English words as for example, the word Jumper, doesn't equate to the same thing. Sweater, is jumper in America. Let's get the functions like accessible buttons correctly labeled edit fields, before we submit this for correction to the English teachers around here. :)

By Daniel Angus M… on Thursday, February 8, 2024 - 21:44

this idea is great, and is baddly needed. ever shop for clothes on Amazon? it's a nightmair. as I use a Vluetooth keybord with my iPhone, holding down space doesn't seem to work. am I doing something wrong?

By Brad on Thursday, February 8, 2024 - 21:44

The thing about language is there's more than just Americans here, we should all matter no matter if we say jumper or sweater.

Anyway, it's not for me so I don't care that much.

Having said that, come on OP, reply to the people who want to use this.

Hell, if you really wanted you could probably contact a lot of stores, ahum, shops, to use this as a feature and might be able to make a bit of cash doing so.

You seamed to have some hope, don't let that die just because my comment was the first you read, there's at least 3 or more people here who want this to be a thing.

What made you think of this? Do you think you could apply this to other things? What's your plans going forward? Let us know :)

By Brad on Thursday, February 8, 2024 - 21:44

Would enabling quicknav help? With NVDA I believe I had to go into focus mode to press space.

Ah, also it might be asking you if it's ok to use your mic and other options like that.