Hi all.
I'm back again with another question, yay! Since I have a new phone with all the memory and storage space which is also technically memory, I've enabled all the AI. Well, specifically I left the defaults alone and then enabled the GPT extension when it said it was ready.
So. What am I doing with it? I know I can make it describe images, "that there's a picture of a goat eating a tin can"! I can get writing advice, which I can live without. What's some other potentially useful or just fun stuff? People keep telling me I need to try the AIs, so here I go.
But I mean, the big things I know about are images, I don't have a lot I need described, and email (both writing and summarizing which I'm not interested in), and honestly, I think that's about it. I'll at least start playing with the image description stuff when I update to 27, I'm still on 26.7 at the moment. But I figure I have everything enabled, so I may as well try playing around with it.
What's it bringing to the table? What might improve my life, what might be fun? My old XR probably couldn't run any of it, so I didn't worry about it. Since the new phone can, well, I may as well try and find out what all the fuss is about. I have it enabled, but I have no idea what to do with it.
I know we have people here who use it regularly, so I'm hoping for some ideas. If there's an Apple page on it, point me at it, I'll do my homework. But I'm hoping for some real world use cases too, not just a generic, it does X Y and Z.
Comments
Wondering the same
Other than image description, I haven't found much of a use for it. I don't need it to write for me either.
Coding and Technical Information
For me, I find AI is most useful for writing code since I have no patience to do it myself, as well as giving me technical information. For example, I used Google Gemini to learn how to use Docker and do various other things on Linux. I'm currently writing a Linux application with Codex.
I generally agree AI is being pushed into far too many things just because it can and companies want to jump on the current fad to make as much money as possible. I'm also concerned about the current problems around information censorship and hallucinations, but I think as long as we use AI responsibly and for specific purposes, it can in fact be very helpful. I've found zero benefits for Apple Intelligence, Copilot, etc, so I utterly ignore all of it.
Re: Coding and Technical Information
I'm using Kodex to build a rugby related website. A few weeks earlier I used claude to make a discord bot for junior and local rugby league. I also use AI to provide summaries, describe images, and give me information about certain phones that I look at.
why I like the AI
I just like it because it gives more detailed responses besides "Here's waht I found on the web." So I'm like you, I use the image descriptions and the chat GPT extention and that's about it.
AI Uses
I use the new Siri AI quite a bit, as I've been testing since the public beta. I love the image description features and use them regularly. I use Siri and the other AI tools for other stuff too.
I don't use it much for writing assistance, but I could see how it is useful for that. However, it is great for getting information from one place to another. For instance, I had purchased an Amtrak ticket a month or so ago. When it was on my screen, I just asked Siri to grab all this information of the ticket on screen, and send it to my Mom via text message. It did it perfectly in about 3 seconds.
I love the ability to quickly and easily find information from old E-mails and text messages. I can just ask Siri to bring up those old Vikings videos I sent to my friend a couple years ago. Instead of scrolling endlessly, I found them in about 2 seconds. Same with old E-mail information or receipts. I was looking for an old receipt E-mail from about 6 years ago, and found it instantly.
I also just ask it general questions, like how to do a task, basic product recommendation ideas, similar song ideas to where I'm at for playing drums, etc. When I was home visiting family last month, my parents were having some weird problems with their Dish TV setup. I just asked Siri, and it gave us good step by step instructions on what to do.
I will likely do more with it, once I figure out to what extent it works with non-Apple third party apps. Unfortunately, my work phone is still an iPhone 14 Pro Max, but I hope to get it upgraded soon. Our work makes us use the Outlook app, instead of the native Mail app. If I can quickly bring up specific E-mails or ask questions about my inbox and other folders, this will be hugely convenient.
I also want to explore the Shortcuts app more now, as it doesn't seem as rigid. I think you can simply describe in your own words what you want a shortcut to do. I also made a post in the New Features for Blind/Deafblind in IOS 27 thread, with an example of how I made a custom Safari extension to make specific websites more VO accessible.
I'm just starting to figure out what I can do with all this new Siri and AI stuff that IOS 27 supports, but it's been very helpful for me, with many practical uses. I hear so many people complaining and dismissing IOS 27 as a not worth while, buggy mess. Yes, there are some bugs, but I don't think they are game breaking, and the performance and these AI features make the upgrade well worth it in my opinion.
Things I might use it for
I develop a lot of bare-metal and close to the metal code, and that includes computer graphics code, so having a local AI that can see and answer simple questions about what I draw can be very helpful. For example right now I'm building a flight simulator to generate procedural virtual environments where I can train tiny models to fly towards objectives and avoid obstacles entirely based on vision, accelerometer, and gyroscope information. Previously I would need assistance from sighted humans to ensure that my math made visual sense, but with a local AI model that I can ask questions about visual subjects I feel almost as enabled when it comes to coding as I used to feel before losing my vision.
Quick chat
SO I just had the following quick chat with the AI after asking it to explore my screen:
Image Explorer Transcript
Description:
A desktop interface displaying a Safari web browser window open to "applevis.com" over a background wallpaper of the Golden Gate Bridge, with a macOS dock visible at the bottom. The webpage lists three forum topics with blue, underlined titles: "Exciting day for visually impaired musicians, music producers, and audio engineers," "i built a free offline camera app that describes what it sees, looking for voiceover feedback," and "norlay," each followed by gray metadata indicating the category, author, comment count, and last update time. A Google Translate banner spans the top of the webpage offering to translate from Portuguese, while the right side of the desktop contains several file and folder icons, including "Nether.MOV.mov" and "MacBook Air Backups."
Question:
Can you tell the center coordinates of the MacBook Air Backups icon?
Answer:
The MacBook Air Backups icon is located at coordinates 954, 661.
Question:
Is this in pixels or points?
Answer:
The coordinates are provided in pixels.
Question:
So the icon is roughly in the middle of the screen?
Answer:
No, the icon is located on the far right side of the screen, roughly halfway down from the top.
Question:
Are you sure you're seeing the high definition version of the image?
Answer:
Yes, the coordinates provided are based on a standard 1000 by 1000 pixel grid, where the top-left corner is 0, 0 and the bottom-right corner is 1000, 1000. The MacBook Air Backups icon is located near the far right edge and slightly below the vertical center.
Question:
So the image that you see is a 1000x1000 pixel square?
Answer:
No, the image is rectangular, but the coordinates are mapped to a standardized 1000 by 1000 grid for reference. The MacBook Air Backups icon is located near the far right edge, slightly below the middle of the screen.
Question:
So the units are not pixels after all.
Answer:
The units are standardized grid coordinates rather than raw image pixels. The MacBook Air Backups icon is located near the far right edge, slightly below the vertical center.
I may write a utility to move the mouse pointer to any arbitrary location on the screen thus avoiding the need for VOCR, but these descriptions sound pretty useful even if not very accurate.
Just realized that I'm posting off-topic
Only noticed that this is the iOS forum now, and I posted about a macOS feature. Sorry about that.
I still think it helps.
I mean, I don't need any of that, but it is a great example of using AI. I like the step-by-step chat transcript too.
I guess I still don't see a ton of usage. As an example, last night I wanted to grab the No Wifi Mini Games app somebody posted about. I searched for it on the app store, had to go here to get the exact game because a ton of other stuff came up, installed it, then picked share to send the link to my wife.
So could I have said, "find no wifi games on Applevis and download it and share it to this contact"?
The thing is I'm not doing coding, I don't see a point where I'd really care about screen layout in that much detail, and I don't have a lot of images I need described.
One thing I do want to do is start doing videos, so I do want to explore help with that, e.g. am I in frame, if I'm playing an instrument can you see it, or if I'm holding it to show it to people, can you see it. Would Apple's AI help with that, or do you think something like Be My Eyes or Seeing AI would do better?
So far image description and coding seem to be the two big things mentioned, but like I said, not really doing either of those. I don't use much social media, I used to be on Facebook, I think my account still exists but I haven't logged on in years now. Memes seem to be a substitute for actual thought, so I don't care about getting any of those described, and like I said, I just don't encounter a lot of pics/images. When I do and get them described, I can't say my life's been all that improved.
So it might just be the case that the things AI does well aren't things I particularly need, at least at the moment. But I'm sure it does other things I'm not thinking about, you know?
Great discussion so far BTW, really appreciate all the responses. Now that I have a phone that's not super slow, I'm trying to do more stuff with it just in general. I figure I should try everything to really be able to decide what works for me and what doesn't. It's the only way to find out really.
Here's a question though, I'm on the free stuff. So can somebody run through a thing with Siri and GPT? Like, what's a thing I can ask Siri that would either hand off to GPT, or .. Let's try it this way, I ask Siri something, I want more info than it's giving me, so what does it look like to make GPT get involved, and keep asking GPT questions instead of Siri? I don't really use Siri except to restart the phone. How do you basically interact with it?