Yo, Human! Scripting Events in the System: Accessing Life with Adaptive Technology

By Nicholas, 4 January, 2018

Previously.

In my previous post, "Mac Scripting, Hello World", we looked at 'tell blocks' where indented commands could be sent to specific applications. The commands were indented one tab spot, while the blocks were finished off with an 'end tell' command. There are several types of blocks, tell blocks, if blocks and repeat blocks to name a few.

I'm Blocking it Out

In this post I wanted to expand on the idea of blocks of code. In particular, a block in a block. An outer block with a double indented block inside. In this case, a repeat block embedded inside a tell block. All of this blockiness has me turning into a block head. I wish I could just copy and paste this stuff into the Editor. Oh wait...

The Web Samples

Using the web samples as a starting point is fine and easily done. You can copy and paste them directly into Script Editor. Paste them into the Script Source text area.

The script samples below start with a Heading called, "Script:". There is a line of dashes at the end of each sample as well. When copying them, do not select the heading, but if you select the dashes it is okay. Omitting the dashes is fine also.

Some of the samples are displayed on the web page in an "as is" fashion which may cause VO to read them as whole paragraphs all at once. VO does not change the structure of the code blocks, except for each Tab spot may be changed to four spaces. Once pasted into the editor, compiling if successful, changes them back to Tabs again.

You can copy them from the web page one line at a time by holding down Shift and using down arrow. Each time will add another line to your selection. When done selecting, press command-c to copy. Note, if you use QuickNav as I do, you can change the Rotor to "Lines" to help in selecting text from a web page.

Or if VO is already speaking the section you want, press Control-Option-Shift-c, to copy last phrase. Even VO's spoken phrase will be formatted correctly, but using four spaces instead of each tab. Compile to make it tabs again. See more about these copy methods farther below.

Lost Down the Block

The title pretty much says it all. Some of my code blocks get pretty long. Plus I end up with sub-blocks that are double-indented, some with triple indents. I finally get to the line I was looking for and then I can't remember what I was going to do here. I often get lost down the block now, without ever leaving my computer chair. :-)

Ignoring the Commentary

Adding comments helps the coder and reader alike. You can add comments to your scripts in two different ways. Comments are completely ignored by AppleScript. When your script is run, it will jump past any comments it finds and continue running your script. You can use comments in any way you wish, as long as you enter them into the script with the proper symbols. For now, lets call them Line Comments and Block Comments.

Adding comments to my scripts sure helps keep me from getting lost. I can add a line comment for quick notes and explanations. I use two dashes and a space to make a one 'wrapping line' comment to myself and others about what the following code block will do. I can quickly type two dashes and make a note as in the example below.

Script:

-- This line has two dashes in the beginning of the line, and would normally tell me what the following code attempts to do. These Line Comments are great for quick notes, for myself and others.
------

Another form of comment begins with a left parenthesis directly followed by a star symbol (shift 8), then a space. This form of comment continues across multiple lines or paragraphs, until it finds the closing symbol, star and right parenthesis. The closing symbol should be on a line by itself as the last line of the comment, as shown below.

Script:

(* This is a block comment. It can contain anything, even multiple paragraphs or whole blocks of code.
  This new paragraph is still part of the above comment, because the script hasn't found the closing symbol yet. But, here it is now on the following line.
*)
------

The Block Comments are great for longer explanations, or introductory info at the beginning of your script. They are even used to temporarily remove whole blocks of code from the running script. This is handy when you are troubleshooting a block of code and you want to confirm that the rest of the script is behaving as expected. One can easily comment out a section of code, test the remainder of the script, then remove the comment symbols to put the questionable block back in again. No need to remove it from the script window, just comment it out temporarily. You can also use empty lines in your script to help separate different areas of the code. Empty lines are also ignored by your running script.

The Evolution of a Script

Many of my scripts start out very basic and look nothing at all like the final versions. This is completely normal and not unlike any other created project that grows over the development process. I usually start with the basic stuff, blocks of code performing the basic functions of my script. Once I get this working, then I add comments and design the 'look' of the script in the scripting window. This helps me realize the flow of things, and gives me reminders to help me remember what I intended for this script, even if it is months before I open it again.

Keep in mind scripts are simply text documents, there is nothing special about them until you Compile. You can feel free to add comments and blank lines, add additional command lines or whole blocks when-ever and where ever you wish. Its just text until you Compile. Even after compiling you can treat it as text again. It doesn't matter as long as you keep the chronological flow in mind. If your blocks are in order, finished off with 'end' statements and the tab indenting looks correct, structure-wise you should be fine. That is not to say there won't be several more glitches while tuning the script.

The next thing I do is enter any delay commands and troubleshoot any delays I had previously entered. Remember, the Delay commands help slow down your script and can be used to customize the script to your particular system. It is also good to have a default set of timing in case your script ever ends up on someone else's system. They can easily find where timing adjustments should be made.

Often I will also enter 'say' commands into my own scripts, at least for my own use. My 'say' commands are usually as brief as I can make them, so they don't mess with the flow of the script any more than absolutely needed. If desired, I can easily comment them out to share with others depending on the situation.

An evolutionary process to build scripts a piece at a time usually works better than trying to create them all at once. Coming back to it in sections and making adjustments seems to end up with a script that is more thought out and better applies to the situation. There are many things to consider when implementing your scripts, like; is the document or app actually in the front right now, are you sending commands to something that is still loading, did you try to show a script dialog without bringing the script to the front, are you sending commands faster than the system can keep up? Fine-tuning a script into place can really make one consider the current state of the interface. It is amazing how many things occur on the screen and with our systems that often go completely unnoticed by the user, any user. Implementing a script into a dynamic running system can be a path of discovery.

Automating Events in the System

Lets dig in now and create a simple script that uses several programming techniques. It will be an automation task that we can add to and let grow as we work. Lets start by having it load Safari and then going to the AppleVis website.

You can copy the scripts in two ways. Set the Rotor to 'Lines' then hold down Shift and use down arrow to select each line, then press Command-c to copy.

Or, VO will read the script as all one line, but you can still use VO-Shift-C to copy last spoken phrase. You can reduce Verbosity levels temporarily to copy just the important stuff. To adjust Verbosity on the fly, press Control-Option-V, then use Arrow keys to adjust settings. Press Escape to stop.

Script:

tell application "Safari"  
    activate  
end tell  
------

Compile and run the script. If Safari loads then we are in business. Now add the following:

Script:

tell application "Safari"  
    activate  
    open location "https://www.applevis.com"  
end tell
------  

The Open Location command allows us to send Safari to a web page using the URL, which is supplied in quotes. In a very 'slam it home' fashion, the above script may work. However, it may not account for timing issues, loading times, current speed of your internet connection, or any of a host of other possible situations. Lets tweak it some with a Delay command.

Script:

tell application "Safari"  
    activate  
    Delay 2  
    open location "https://www.applevis.com"  
end tell  
------

Now the script may give Safari a chance to load before telling it to go somewhere. We could easily leave it at this point and call it good. But that's just not how we roll. Lets add in some verbal responses.

Script:

say "opening Safari."  
tell application "Safari"  
    activate  
    delay 2  
    say "loading AppleVis."  
    open location "https://www.applevis.com"  
end tell  
------

That's better, but never enough. :-)

The first thing I normally do on the AppleVis web site at this point is move VO down four headings, just above the "Latest Posts And Updates," section. I usually use Quick-nav for this, but I could easily use Control-Option-Command-h, to move to the next heading four times and end up in the same place.

An Eventful System

There is an application one can address from inside a script, called "System Events." With it one can send keyboard, mouse, and other events to the system. It allows me to have my script press shortcut keys. Using the 'keystroke' command, I can send keyboard events to the system from my scripts. The sample below shows how to keystroke the Save command shortcut, command-s from inside a script. If you run this from Script Editor it will try to save the front script, just as if you had pressed the keys yourself. You could use scripting to pull any other app/document to the front, then issue the save command and it would attempt to save that document instead.

Script:

tell application "System Events"  
    keystroke "s" using command down  
end tell  
------

All the above script does is press Command-s. It doesn't account for anything else, if the keys don't actually apply right now, oh well! You could make it care about more if you wish.

The next small script shows a repeat loop block. They can be real handy in a script. They are often buried in other blocks to help facilitate repetitive commands.

Script:

repeat 2 times  
    say "Hello world!"  
    delay 2  
end repeat  
------

Okay, I am obviously leading up to something, Let's use a repeat loop to press Next Heading for us, four times. The headings part requires that VO is running. Without further delay, (cough), here is a script that opens Safari and moves to the AppleVis web site and then moves four headings down, complete with comments, delays and verbal progress reports.

The most direct method for copying the script below is to get VO on the top line, then hold down Shift and use Down Arrow to select the lines. If you use Quick-Nav, change the Rotor to Lines first. Many of the lines below start with two dashes and a space, to indicate a comment. Make sure you select these as well to reduce errors in the Script Editor.

Script:

-- the block below opens Safari, or brings it to the front. Then opens the AppleVis website.  
Delay 1  
say "opening Safari."  
tell application "Safari"  
    activate  
    delay 3  
    say "loading AppleVis."  
    open location "https://www.applevis.com"  
end tell  

-- delay for a bit and let AppleVis load.   
delay 3  

-- the block below sends the 'next heading' command from VoiceOver, to System Events, 4 times in a row as if you pressed them yourself.  
-- it uses a repeat loop that is sub-indented inside the tell block.  
-- it will finish its own repeats before going on to finish the tell block.  

say "Heading down to recent posts."

tell application "System Events"  
    repeat 4 times  
        beep  
        keystroke "h" using {control down, option down, command down}  
        delay 1.5  
    end repeat  
end tell  
------

The keystroke is issued from inside the repeat loop, embedded inside the tell block that addresses System Events. The repeat block itself is indented one tab spot, while the commands inside it are indented two tab spots. The end repeat command is indented only one tab again, then 'end tell' is not indented at all.

The keystroke command can issue a single shortcut key, or use a list of modifier keys inside of braces. Between both methods you should be able to automate as many keystrokes in a row as you wish.

Get the above script into Script Editor and successfully compiled. Then run it to watch the whole thing happen. You can use its pieces to build a better script, you have the technology! :-)

A Textual Event

Another use for System Events, among many others, is to enter text. You can 'keystroke' text into any document or text box. Simply supply the text in quotes, as in the following script.

Script:

tell application "TextEdit"  
    activate  
    delay 2  
    make new document  
end tell

tell application "System Events"  
    keystroke "Here is the AppleVis web site."  
    keystroke return  
    keystroke return  
    keystroke "www.applevis.com"  
end tell  
------

The above script loads TextEdit then makes a new blank document. It then inserts the first line of text, then presses the return key twice. Then it enters the web address for AppleVis. It could just as easily entered passwords, whole paragraphs, or any chunk of text. The possibilities are only limited by your imagination.

In Conclusion

So now you can easily have your scripts press shortcut keys for you, as well as enter text. There are many things that System Events can do and additional ways to enter keys. Some keys have code numbers instead of letters and require the "key code" command. Some situations call for holding a key down through out several commands, then releasing it again. System Events can also allow use of mouse control and other events that one would normally activate themselves.

Dabbling with AppleScript can be a very mind blowing experiment, greatly expanding one's knowledge of how their Mac actually works. One more thing it always seems to remind me of...

All of our cool digital stuff that we work with, play with and enjoy, is all about "Living." Live well!

Portions copyright Apple, Inc. All rights reserved.

Options

Comments

By Callum Stoneman on Friday, September 21, 2018 - 22:21

Hi,

Thanks so much for posting this! I'm just starting to play around with Apple Script and I'm just creating some fairly simple ones to save me a couple of keystrokes here and there for now, so these examples, as well as the ones in your previous post on this, are exactly what I was after.

When I copied your script for opening Safari, going to the AppleVis website and then jumping down to the "latest posts" heading, everything on that script works apart from this line:

keystroke "h" using {control down, option down, command down}

Everything else works, including the beep, but it doesn't seem to perform the keystroke properly. I'm having a similar experience with a script I'm trying to create that will open an application and jump me to the item saved as VO hotspot 1, which is control option 1. It will do a couple more things as well but this is as far as I've got for now. The application I'm trying to use is Djay Pro, but I've changed the code slightly so it will use an application that is definitely on your Mac. First, go into Safari and save something as hotspot one by navigating to it and pressing control, option, shift and one, then try running this:

say "Opening Safari"
tell application "Safari"
activate
end tell
delay 3
say "Pressing VO 1"
tell application "System Events"
keystroke "1" using {control down, option down}
end tell
say "Script completed, was it a success?"

Well, the script asked my question for you. :) was this successful at your end? For me, everything runs apart from it pressing VO 1. Any ideas as to what may be going wrong?

Thanks for your help and sorry for such a long comment.

By jrjolley (not verified) on Friday, September 21, 2018 - 22:21

Enough is enough, I have to comment now. Why do all of the blogs here at Applevis insist on having daft funny comments in them? I come here as a reader, not for bloody stand up. It's actually very annoying and rather immature to be honest to be associated with you lot because all you seem to do is joke and make daft remarks in content. There's no need for it. If this site is to be taken seriously, which is isn't by the way by real iOS users, quit with the quippy funny comments and write properly. How this site has a blog team with people who don't know how to produce a technical article is beyond me.

By Callum Stoneman on Friday, September 21, 2018 - 22:21

James, personally I'd say someone throwing a strop is more immature than a few funny comments to make the reading more interesting, but that's just my opinion. Oh and by the way, if this site isn't taken seriously enough for you, no one is forcing you to be here.

By jrjolley (not verified) on Friday, September 21, 2018 - 22:21

In reply to by Callum Stoneman

I don't come to a serious site regarding Apple products to have pathetic quippy comments in my content thank you very much. This site is a joke. No matter how many comment threads I attempt to engage in, how much advice I offer users, it all falls on deaf ears. I can tell this is a blindness site because most of you are the same. You don't research anything for yourselves playing on email lists like it's 1989. You'll all whine and complain about the littlest accessibility issue and get comfort from other whiners. I've had it. Seriously.

This pathetic excuse for a site's not even designed properly to work with iOS - braille screen input doesn't let users enter blank lines as an example.

In conclusion then, Applevis is meant to empower people, I have had more empowerment by reading content from sited authors who understand what they are doing instead of playing on their devices. Criticism here is taken as offence.In order for people to take us seriously as blind iOS users, stop with the daft comments, write technical articles in the mannor to which it is expected and take criticism as it is.

By Nicholas on Friday, September 21, 2018 - 22:21

Hello Callum,

Thank you for your comment, I am glad my post is helping you explore AppleScript. Looks like you're getting a handle on it nicely.
I looked at the code sample you provided and got the hotspot thing to work using the following line:

keystroke "1" as integer using {control down, option down}

When you keystroke things inside quoted text, AppleScript accepts it as plain text. Use the "as integer" phrase to tell your script that you mean the integer version of 1, not the plain text version of "1". You can also designate quoted material "as text" and "as string".
I retried my original script for "next heading" as well, it seems to work as expected on my system. The first thing I would look at, is the VO keystroke still set to move to next heading? I check by turning on VO's keyboard help using VO-k. Then press the keys you are trying to use and make sure VO reports the correct function. Press escape to exit keyboard help.
If that checks out, try the keys manually yourself in Safari to eliminate some Safari shortcut interfering as a possibility. If that checks out, I would look at timing issues next, Safari takes longer on a fresh load of the app, than just a fresh load of a webpage. Increase or add delays where you think are needed to see if it helps.
If the "next heading" thing still won't work, you could use your method and place hotspot 2 just above the recent posts list.
Thanks again for the comment, go for it! :-)
Best regards.

By Nicholas on Friday, September 21, 2018 - 22:21

Hello James Jolley,
You know, I have spent the last few days trying to find an appropriate, open minded response to your comments above. After much consideration, I have decided that there isn't one. Trying to address the many issues that you describe would be completely off-topic.
There is one mistaken idea that you present above, that is relevant to my blog in general, though it is not really on topic for this particular post. For clarity sake, I will address it anyway. This blog was never intended to be a "technical article," of which there are many accepted formats. My blog was always intended to be 'a light hearted, sometimes humorous look at life with adaptive technology.' The venue for my blog was never meant to be a "technical article," as you describe it above. I would think that would be apparent from the title, from the format, or from the content. So all of your venting above was based on a mistaken concept from the start.
I agree with the general idea, that tech articles as you call them, should not contain frivolous information. I have written hundreds of tech articles, outlined reports on interface complexities, complete dissertations on an entire programming language and many software tutorials, as I was hired to do. I started on a professional level more then twenty five years ago. All of it was formatted for the proper venues, as stated in the contracts. That, however, is also off-topic.
Aside, I truly hope you find the information you are looking for. I wish I could say it was nice to meet you, but...

By Callum Stoneman on Friday, September 21, 2018 - 22:21

Hi Nicholas,

Thanks for your response. I tried editing the code to include "as integer" as you suggested, however that didn't seem to work either. The only difference it did make was that I now hear a beep when that keystroke is being pressed, even though "beep" isn't part of the code, which makes me think the system is pressing a slightly different keystroke to what I'm intending somehow. Is there anywhere else I can view errors that have come up in the script that are not showing in the editor when compiling and running?

Regarding your suggestions with your script above, I tried all that before I posted the reply. Pressing VO +cmd +H manually works fine, and all the VO settings are correct, but still no luck. The joys of programming! :)

Just a thought, what version of MacOS are you running? Could there have been a slight change in updates? I'm on the latest version of High Sierra.

Hi Callum,

Sorry for my delay in responding, life is what happens when you're busy making other plans. I am using latest High Sierra also. Ah, the wonderful world of trouble-shooting. :-)
A few things to check;

In VO Utility, (VO-fn-f8), in the General category, down near the bottom of the window,
Allow VoiceOver to be controlled with AppleScript, checked.

In System Preferences>Security & Privacy>Privacy tab>Privacy Categories table>Accessibility:
it states "Allow the apps below to control your computer." next is a table of apps that are being allowed. If you don't see Script Editor in the table, click the Add button beyond the table. This should give you an open app type window. Choose Script Editor to add to the table.
Script Editor can be found in:
Applications folder>Utilities folder>Script Editor.

This may be needed to facilitate GUI scripting by running scripts from the Editor. Not sure, this changed a an OS or two ago.

Both of the areas above may allow for scripting the GUI. One allows Script Editor to run script containing GUI commands. The other allows scripts to control VO, which may be considered GUI scripting (not sure), but it marginally has to do with System Events.
Either or both of these items may help. If these settings do not help...
Another possibility, is Safari or the target web page not in focus when you address system events for some unknown reason? Receiving the system beep usually means the keystroke does not currently apply. It may be a focus issue.
Possibly, something else is grabbing focus from Safari before System Events starts issuing keystrokes. When addressing System Events as its own 'tell block', the keystrokes are issued to the currently-in-focus app. If Safari or a webpage gets pushed into the background for any reason, the issued keystrokes may not apply.
I have seen occasions where my script would load Safari in the background and I couldn't figure out why. In my Safari tell block I used the activate command twice, once to load Safari, then again to bring it to the front. Then send my keystrokes to it.

tell application "Safari"
activate
delay 0.5
activate
end tell

Then address System Events and issue the keystrokes.
You may realize this already, but for general knowledge;
When you run scripts for automation purposes, since they depend on the system to be in a certain condition, never click on anything on the screen while it is running. It is easy to accidentally mess with what the script expects to be there.
Another good approach is to put redundant commands on areas that are being stubborn. in a line above your System Events block, just to make sure Safari is in front, put this one-liner:

Tell application "Safari" to activate

This does the same as the tell block, but for single commands, you can enter them as one line using the word "to". No need for an 'end tell' at the end. The only stipulation is that the entire statement fits all as one line. Indented lines underneath are not allowed. You can go back and forth using both single-lines or indented blocks, as needed.
Note: telling safari to activate if it is already running will simply bring it to the front. Same with all apps.

One more thought, how is your VO modifier set? I tried using CapsLock as the modifier and it was causing issues with my use of "VO keystrokes". I had to change it back to Control-option in order to use those keys in my scripts. Just a thought.

As for errors, I use the extra window below where you type source text. There are three tabs, the log tab can sometimes give more info, sometimes not.

If all of this is to no avail, then we can start digging deeper. If you discover what the issue is, or that it still isn't working, let me know.
I hope you have success with it. And keep in mind, there are many possible approaches.
Best wishes !

By splyt on Sunday, October 21, 2018 - 22:21

I have been pationate for programming cinse very early in life.

Specially when it comes to blind related mathers I can easily see the huge difference software has the potential to make in our lives. In fact, screen readers will give access to a whole bunch of oportunities that blind people would hardly have otherwize ... really.

Ironically, most screen readers were designed and first coded by blind people, you know ... we know what we need and nothing for us without us and all alikes.

I imagine how many other great solutions for blind people could have been developped if we had more blind developpers working on it. I imagine how many great solutions in general we could have if more people ... sighted or not ... would become developpers.

Because of that, Nicholas, I congratulate you. Because programming does not have to be a monster unless you are coding an operating system and, even so, it does not need to be a monster for those persons who are used to it. Programming should be thaught with humor and lightness to enable people to come up with very creative solutions for their problems.

Long time passed are the ages of c++ and assembly, technologies I am familiar with by the way, where the common of persons shouldn't be around in programming. I am a huge intusiaste of making programming a dimistified thing and I think you are going through the right way.

By Callum Stoneman on Sunday, October 21, 2018 - 22:21

Hi,

Well, I've found the problem that was causing my scripts to not run properly, the "Allow VoiceOver to be controlled with Apple Script" option was unchecked, and Script Editor wasn't in the list of apps that could control my computer. It's been so long since I've had to open VO utility I'd forgot that option even existed! :)

I have added Script Editor to that list no problem, but I can't enable the "Allow VO to be controlled with Apple Script option". When I check the box, it asks for my password, a "busy progress indicator" appears next to the checkbox for a few seconds, then it reverts back to being unchecked. Again no errors or anything that I can see. I have added VO utility to the applications that can control my computer, but this still doesn't enable it.

Regardless of this, the scripts still work as expected! Thanks so much for your help with this, Nicholas.