iOS Camera App Hallucinating

By OldBear, 26 May, 2024

Forum
iOS and iPadOS

I was trying to take a picture of a wooden toolbox from a few differen angles yesterday. During this, I had to adjust things in the box, and had the VO focus on the view finder. The description changed from a wooden box with various tools to a person holding a wooden box when my hand entered the frame.
This might be useful, but the descriptions always started with information like, natural lighting, and a few other things that got in the way of a continuous description. Then it started hallucinating a bunch of weird things. So I covered the lens with my hand to shut it up for a minute, and it really started hallucinating, like a room full of people, a screen shot of black and white letters, etc, etc. I can't imagine what it was receiving as information to come up with any of that.
Is this the kind of hallucinating that the other AI apps and devices are doing? It does it in a dark room too. Reminds me of what a sighted brain does when staring into pitch blackness.

Options

Comments

By Brian on Saturday, May 25, 2024 - 03:22

I take it to mean that you are using the "Live Photo" feature? From what I have experienced trying to use that, I am under the impression that both cameras are being utilized at once. Allow me to explain. . .

I was trying to use this in my living room, to see what it could pick up, and I kept getting results like; "A wide screen television on a stand", " a table with a glass top" (my coffee table), "a small couch with framed pictures on a shelf above it" (I actually have a bookcase with framed pics on it, behind said couch), anyway during all of this, I would also get random things like, "a man's face", "a man taking a selfie", "a man sitting on a couch holding a smart phone".

Anywho, the point is I was holding my iPhone straight in front of me, with the screen facing me, and the back camera facing away and scanning my living room area.

This is why I believe that both cameras are being utilized, and also likely why hallucination occurs.

To answer your question, however, I do not think it is exactly the same as the LLMs usually only have a single camera/image they are working with.

What I think might be happening, is that perhaps multiple samples are being utilized behind the scenes, think HDR, and that could be causing hallucinations.

Just my thoughts... 🤷

By OldBear on Saturday, May 25, 2024 - 03:22

Live Photos are off. This is just with VO focus on the camera view finder.
I have had the view finder pick up and describe my reflection off glass and shiny metal. I tried this in a completely dark room and it was seeing a bunch of stuff that wasn't there, much less there being no light to see anything.
I don't recall it hallucinating before, but maybe it's trying so hard to see something that it generates archetypes from raw static or something.

By Yvonnezed on Saturday, May 25, 2024 - 03:22

Makes you wonder if a lot of hallucinations could be prevented if programmers trained these things to admit they didn't know rather than inventing answers,, 😀

By OldBear on Saturday, May 25, 2024 - 03:22

It would have been useful to me if it said, in some form or another, there was no light or something is blocking the view. That seems to be when it starts making up things. However, it might also just be some flaw in my phone, and almost no one else gets this effect. Completely possible with Apple.

By Brian on Saturday, May 25, 2024 - 03:22

Well, since you are not using Live Photo, and are taking pics in completely dark rooms, there is only one logical answer.

You are being haunted. 😨

Thank you, that is all. 😀

By Ollie on Saturday, May 25, 2024 - 03:22

This is different to LLM hallucination. Where an LLM actually creates data for its hallucination, Queen Elizabeth Played bass, the best way of keeping meatballs on a pizza is with glue, and Santa isn't real... This is merely a failure in pattern recognition. IE, the image recognition here isn't generative and so this is simply an error rather than, at least in the LLM lingo, a hallucination. The LLM will literally see things that aren't there, this is just misidentifying objects.

By OldBear on Saturday, May 25, 2024 - 03:22

I'm not sure what objects the camera is detecting to misidentify when I've covered the lens with my hand, or the room is completely dark. It's like if it starts looking through its own patterns when it has no external information.

By Holger Fiallo on Saturday, May 25, 2024 - 03:22

Maybe the camera is using lSD?

By Ekaj on Saturday, May 25, 2024 - 03:22

I've had this happen a few times too, and it's interesting. On a related note, is the term hallucination taken from what happens to the brain when somebody is on recreational drugs or something like that? In any case, I turned off the switch that says something like Precise and was able to get NaviLens to read the word "Kellogg's" on a box of cereal bars over the weekend.

By Ollie on Saturday, May 25, 2024 - 03:22

Most things we experience are hallucinations. It's really interesting, it's known as predictive processing, which is actually very similar to what an LLM does. Basically, the brain creates the world around us based on limited data and only corrects the prediction if needed. We basically see what our brain expects and so things that don't fit into the usual pattern of life, will either be ignored, the famous example of a man dressed as a gorilla passing through a party that no one noticed, or, if it is considered a threat, the brain flags it and recomputes the world.

This is, of course, making it sound really simple and like we go around experiencing nonsence all the time which is demonstrably wrong otherwise we'd fall apart. It's highly sophisticated prediction that is constantly training itself. I'd look into it if you find such things interesting.

By PaulMartz on Saturday, May 25, 2024 - 03:22

I agree with Ollie.

For Old Bear, a better example of AI hallucination would be pointing the camera at your washing machine and asking what the knobs are set for, and the AI would reply confidently that it's set for hot water normal wash, when in fact it's set for cold water delicate wash.

So I tried to reproduce what Old Bear reported. I opened the camera and put VO focus on the image, hoping to hear an image description. Instead, I experienced behavior I've never encountered before. VoiceOver repeatedly said, "point five one times, point five one times," regardless of where I aimed the camera. Or it would tell me tilt left or tilt right, or it would say zero people. But I never got an image description, even though image descriptions were on.

It was the "point five one times" thing that really baffled me. Anyone know what that is? I've never heard it before.

Anyhow, with all this babbling, tilt left, zero people, etc., the last time I tried to use the camera to scan a QR code, I found it almost impossible. Just as it announces, "QR code detected," before I could double tap, it would go off again about zero people or tilt right. Does anyone else use the camera with VO enabled to scan QR codes? If so, what's your technique?

By OldBear on Saturday, May 25, 2024 - 03:22

I don't know a whole lot about AI or what's going on with the Camera App describing what is in the view finder, so I'm not surprised I used the wrong term. I'm also not surprised that your Apple device has different behavior from mine, Paul. That seems to happen on Apple devices.
The only important thing I can point out is the effect happened when I covered the lens with my hand, so no light input, and when I took the camera into a completely dark room. And to Ollie's point, that's a bit like what a sighted person's brain does in darkness, I suppose. I wonder if the phone gets like phantom keyboards and things when you disconnect them... Might explain a lot... LOL

By OldBear on Saturday, May 25, 2024 - 03:22

@PaulMartz, that sounds like what the zoome feature says. It is a big process to get the zoom option to come up on the rotor with Voice Over on, doubt you did that, and it does not repeat itself when I have focus on the view finder when I tested it on my phone: SE 2022. It only says it when I swipe up and down with the rotor set to zoom. No idea why yours isn't describing, if it is describing images in other apps. I had to turn it on globally once in VO settings once after an update to get it to work, but other than that.
All that zero persons and natural lighting information coming befor the image description does get in the way. However, it will tell me if it sees a face, rather than zero persons, and I use that a lot to map out where the edges of the frame will be when I'm setting up my tripod. Mixed bag, I guess.

By Ollie on Saturday, May 25, 2024 - 03:22

I think OldBear's experience and the issues I've got in the photo album in not being able to flick through images, has soured my experience of IOS photos. The frustration is I think that its one of those things people don't use much and has little work done on it.

Regarding QR codes, wave it till it works... It's very frustrating. I think the more recent phones are better at it with their wide angle cameras, but I use the code scanner which you can add to your control centre rather than the camera app which, as stated, is a bit wonky, either giving you too much information that is incorrect or none of the information you desire... Which rates it less than useless.

By jim pickens on Saturday, May 25, 2024 - 03:22

At Ollie
Could you elaborate further on this, as well as provide a few links for further explanation? I’m intrigued.

By Ollie on Saturday, May 25, 2024 - 03:22

Really off topic, but the book: Being You , By Anil Seth delves into it as well as measurements of consciousness. It's very interesting, basically saying that all consciousness is an hallucination based on complexity of the host and the data which may be observed, which raises some very interesting questions.

But yeah... Apple products... Yup.

By OldBear on Saturday, May 25, 2024 - 03:22

From what I've read, we are... simulating a world and a sense of self and body in that world., and that's from a bunch of information, including culture and things like beliefs etc, and even genetic expressions. It seems to also be shared and communicated between people, like networks of brains, and who knows what else. I'm not hearing any thing that sounds like AI or LLM is doing that. At least not yet. Do I have that wrong?

By Ollie on Tuesday, May 28, 2024 - 03:22

The generation of perception through biological input and predictive generation of data based on past experience/a data set, is correct. The second bit, everyone being connected together in a hive mind, I believe, is dipping into meta-science. It may well be true, but there is no reviewed scientific data that supports it. At least, as far as I know. I know people like Elon Musk believe that the world is an hallucination but I'm not sure how that deals with the self and any bridging between people. As far as i understand it, consciousness is restricted to the system which generates it, IE, your body.

This, of course, is all a little disquieting too, especially for those with faith in a higher power. I'm certain that there are some very big minds that can give a solid counterpoint to this rather cold analysis of who and what we are.

By Brian on Tuesday, May 28, 2024 - 03:22

Disclaimer: Why yes, I did steal that for my subject line. Accept this!

So if we are all hallucinating/simulating our reality, what are we doing when dreaming?

I have had some of the most vivid, intense, and at times terrifying dreams in my life. Please don't tell me those are real. 😟

By Ollie on Tuesday, May 28, 2024 - 03:22

I think it's more that nothing we experience is actually 'real'.

HTH

By OldBear on Tuesday, May 28, 2024 - 03:22

Sorry about that, Ollie, hive mind isn't what I meant, and I can see how what I wrote sounds like that. I was saying we can influence and change each other's perceptions and internal models of the world to be similar by communicating, whether consciously or unconsciously. Not that a group of people have a shared consciousness, or that consciousness can escape the brain etc. More that many people can have very similar simulations going on in their brains at the same time. It was the sense of self I was wondering about with AI.
As a couple of extremes, however, I read something about certain conjoined twins with shared brain tissue possibly having shared thoughts, and of course, there can very nearly be two consciousnesses in a person when the corpus callosum is cut and any other external paths of communication between the two sides of a person's brain are blocked.
@ Brian, you are a butterfly dreaming you are a human, or is it the other way around? (Zhuangzi)
* That edit is a little better.

By Ollie on Tuesday, May 28, 2024 - 03:22

Ah, with you. Yeah, I guess we could say there is no such thing as a closed system, we act and react.. Or maybe just react, to each other. Meta concepts, as in conventions of language, law, etc, have been built in this way.

I'm a dream butterflying I'm a man.

By Dominique Stansberry on Tuesday, May 28, 2024 - 03:22

Like the subject says, replying to the person that spoke about dreams and asked what do we do when sleeping...
Heere's some text from what I'm working on...
Enjoy...
*grin*

The Submarine Journey:
The chill of the night air was a stark contrast to the warmth of my bed where I lay drifting between consciousness and sleep. It was during one of these liminal states, in late 2018 or early 2019, that I experienced a dream unlike any other—a dream that would become a pivotal moment in my journey of self-discovery.
Aboard the Submarine
In this dream, the boundaries between the waking world and the realm of the subconscious seemed to dissolve. I found myself aboard a submarine, navigating through its confining space with a sense of purpose I couldn't quite understand. The realism of this experience was unmatched by any dream I'd had before, prompting me to question my destination and how I arrived there. It was my first encounter with lucid dreaming, where the dream felt as real as waking life.
Arrival in a Place of Peace
Upon reaching my destination, I was greeted by an environment vibrant with life—lush trees, abundant plants, and an overwhelming sense of peace. This tranquility made me yearn to stay indefinitely, connecting with a part of myself that longed for understanding and belonging. Remarkably, in this dream, my blindness did not limit me. I could see everything around me with clarity, feeling an unprecedented connection to the world I had stumbled upon.
The Reluctant Return
Despite the serenity, a sense of urgency began to take hold—the feeling that I needed to escape, to return to my physical existence. Boarding the submarine once more, I felt my consciousness begin to slip away as I made the journey back to my body, my heart heavy with sadness for leaving that peaceful place behind.
Awakening and Revelation
This dream marked a significant turning point for me. Previously, I had viewed dreams with a certain indifference. However, the emotional depth and vividness of this experience ignited a newfound curiosity within me about the messages hidden within my dreams. It wasn't until I encountered the TV show "Atlantis" in 2021 that I made a startling connection: my dream closely mirrored the narrative of Jason's journey to Atlantis, hinting at a past life that had been reaching out to me through the veil of sleep.
This revelation deepened my understanding of my spiritual journey and the bonds I share with figures from my past. It highlighted the intricate ways in which my past lives continue to influence and shape my present reality. As I navigate the complexities of these discoveries, I find myself on a path of exploration—a path that delves into the themes of reincarnation, spiritual awakening, and the eternal quest for self-discovery and understanding.

https://tappedin.fm/archives/45/tappedin/audio/ep9-lucid-dreaming-and-spm/

By OldBear on Tuesday, May 28, 2024 - 03:22

@Dominique Stansberry, I wish you success with your writing. I wrote down my experiences, long, long ago, in a placenot at all far away, after an unexpected reaction to coming off a depression medication and having to figure out what happened on my own. I have no desire to publish my writings... They are nettles for wild swans, or some other allusion of yore, but do make good notes for my world view when someone asks... or shoves.
@Ollie and Brian, James Fallon has a brain scan for that...

By Dominique Stansberry on Tuesday, May 28, 2024 - 03:22

Sutch things as QHHT and Hypnosis help with such conclusions.
I will say this,
lot's of what you may think isn't real, actually isn't made up at all. I've found that out the hard, and not so hard way...
I love the journey though.
Thanks for the well wishes.