Two finger swipe up gesture behaving in consistently in Safari, voice over

I’m just having a strange issue at the moment with iOS and voice-over, when are use the two finger swipe up gesture in Safari, sometimes it will jump to the top of the page which is supposed to do.
But most of the time, it will jump up to the back button, which means I have two swipe through all the options before I can get to the top of the page.
To make things worse, sometimes those options are at the bottom of the screen instead, and so I can’t get to the top.
Has anyone else experienced this issue.
Do you think it might be a bug, or is there a setting I can change?