improve mobile unlock code and chapter to readme

Author: chrisweb

README.md

@@ -222,7 +222,7 @@ Note: if you use typescript, import the **ICoreOptions** interface along with th
 * playNextOnEnded: [boolean] (default: true) when a sound or song finishes playing should the player play the next sound that is in the queue or just stop playing
 * stopOnReset: [boolean] (default: true) when the queue gets reset and a sound is currently being played, should the player stop or continue playing that sound
 * visibilityAutoMute: [boolean] (default: false) tells the player if a sound is playing and the visibility API triggers a visibility change event, if the currently playing sound should get muted or not, uses the [Page Visibility API](https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API) internally
-* createAudioContextOnFirstUserInteraction: [boolean] (default: false) for a sound to be played the player needs to have an audiocontext, on mobile you can NOT play sounds / songs until the user has interacted in some way with your UI, this means autoplay with no user interaction will get prevented by the mobile browser (iOS (iPhone) and android mobile devices, today iPad is considered a desktop device and does not need this), when this option is set to true the player will try to catch the very first user interaction and initialize and audiocontext so that when a sound needs to be played the context will be available, there is a second option called "unLockAudioOnFirstPlay" which will do the same thing but instead of doing it on the first user interaction it will do it on the first call to play(), this is the preferred method as it keeps the audioContext turned of until it is really needed, which is preferrable to save resources on mobile
+* unlockAudioOnFirstUserInteraction: [boolean] (default: false) this tells the player to attempt to unlock audio as soon as possible, so that you can call the player play() method programmatically at any time, for more about this check out the chapter ["locked audio on mobile"](#locked-audio-on-mobile), if you don't want to the player to handle this part and prefer to do it manually then you can use the [player function](#player-functions) called **manuallyUnlockAudio()**
 * persistVolume: [boolean] (default: true) if this value is set to true the player will use the localstorage of the browser and save the value of the volume (localstorage entry key is **WebAudioAPIPlayerVolume**), if the page gets reloaded or the user comes back later the player will check if there is a value in the localstorage and automatically set the player volume to that value
 * loadPlayerMode: [typePlayerModes] (default: PLAYER_MODE_AUDIO) this is a constant you can import from player, currently you can chose between two modes, [PLAYER_MODE_AUDIO](#player_mode_audio) which uses the audio element to load sounds via the audio element and [PLAYER_MODE_AJAX](#player_mode_ajax) to load sounds via the web audio API, for more info about the modes read the [modes extended knowledge](#modes-extended-knowledge) chapter
 * audioContext: [AudioContext] (default: null) a custom [audiocontext](https://developer.mozilla.org/en-US/docs/Web/API/AudioContext) you inject to replace the default audiocontext of the player
@@ -330,6 +330,37 @@ player.addSoundToQueue({ soundAttributes: mySoundAttributes })
 * getVisibilityAutoMute() get the current boolean value that is set for the **visibilityAutoMute** option
 * disconnect() disconnects the player and destroys all songs in the queue, this function should get called for example in react when a component unmounts, call this function when you don't need the player anymore to free memory
 * getAudioContext() get the current audioContext that is being used by the player [MDN audiocontext](https://developer.mozilla.org/en-US/docs/Web/API/AudioContext)
+* manuallyUnlockAudio() this method can be used on mobile to unlock audio, because by default audio won't play on mobile if the code that triggers the player play() method is not a user interaction, for example a user pressing a play button works fine, but if you want to play a sound programmatically without any user interaction then the mobile browser will throw a notallowed error, this method can be used to unlock audio on a user interaction, after audio is unlocked you can use the play() method programmatically, an alternative if you don't want to implement this yourself is to enable the [player option](#player-options) called **unlockAudioOnFirstUserInteraction**, for more about this check out the chapter ["locked audio on mobile"](#locked-audio-on-mobile)
+* checkIfAudioIsUnlocked() function you can use to check if audio is unlocked on mobile, for example after calling manuallyUnlockAudio()
+
+### locked audio on mobile
+
+On mobile you can NOT play sounds (songs) programmatically until the user has interacted with the website / webapp
+
+If the user clicks on a play button and in your event handler you call the play method of the player then everything is fine as it is a user interaction that triggered the play method
+
+However if you want to play music programmatically on mobile, then the page / app needs to be "user activated", in other words on mobile when a page has finished loading audio is still locked you will not be able to programmatically play a sound (song), the play method will return a **NotAllowedError** error:
+
+> The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission (No legacy code value and constant name).
+
+This means that autoplay without a prior user interaction will get prevented by the mobile browsers
+
+iOS (iPhone) and android mobile devices will throw an error, in the past iPad tablets would throw an error too, however newer versions are considered a desktop device and do not need throw an error
+
+The process of unlocking audio on a mobile browser is called "Transient activation" and the player can help you achieve this if you don't want to write your own code
+
+* solution 1: there is a [player option](#player-options) called **unlockAudioOnFirstUserInteraction**, set it to true when initializing the player and the player will add user interaction listeners to the html document, on the first user interaction the player will attempt to unlock audio, after audio is unlocked you will be able to call player play() function programmatically and it will not throw an error anywore
+
+* solution 2: there is a [player function](#player-functions) called **manuallyUnlockAudio()** that you can use to attempt to unlock audio on mobile, this function MUST be played inside an event handler for a user interaction like "keydown" (excluding the Escape key and possibly some keys reserved by the browser or OS), "mousedown", "pointerdown" or "pointerup" (but only if the pointerType is "mouse") and "touchend"
+
+After audio got unlocked, you can use the [player function](#player-functions) called **checkIfAudioIsUnlocked()** to check if now audio is unlocked, this is also useful because the browsers have an internal timer that starts running after "Transient activation" happend, so if the website (app) is idle for a while the browser might lock audio again, meaning you need to re-unlock it again, what the exact duration of that timer is depends on the browser and is not something browser vendors make public (in might also be different depending on the version of the browser)
+
+Read more:
+
+* [MDN: Features gated by user activation & Transient activation](https://developer.mozilla.org/en-US/docs/Web/Security/User_activation)
+* [WebKit: The User Activation API](https://webkit.org/blog/13862/the-user-activation-api/)
+* [MDN: Navigator: userActivation property](https://developer.mozilla.org/en-US/docs/Web/API/Navigator/userActivation)
+* [caniuse: Navigator API: userActivation](https://caniuse.com/mdn-api_navigator_useractivation)
 
 ### modes extended knowledge
 

dist/index.js

@@ -31,6 +31,7 @@ export declare class PlayerAudio {
     protected _audioElement: HTMLAudioElement;
     protected _mediaElementAudioSourceNode: MediaElementAudioSourceNode;
     protected _isAudioUnlocked: boolean;
+    protected _isAudioUnlocking: boolean;
     constructor(options: IAudioOptions);
     protected _initialize(): void;
     getAudioNodes(): IAudioNodes;
@@ -39,6 +40,7 @@ export declare class PlayerAudio {
     protected _addFirstUserInteractionEventListeners(): void;
     protected _removeFirstUserInteractionEventListeners(): void;
     unlockAudio(): Promise<void>;
+    isAudioUnlocked(): boolean;
     protected _createAudioElementAndSource(): Promise<void>;
     protected _createAudioElement(): Promise<void>;
     getAudioElement(): Promise<HTMLAudioElement>;

dist/index.js.map

@@ -21,7 +21,6 @@ export interface ICoreOptions {
     loadPlayerMode?: typePlayerMode;
     audioContext?: AudioContext;
     addAudioElementsToDom?: boolean;
-    unLockAudioOnFirstPlay?: boolean;
 }
 export interface ISoundsQueueOptions {
     soundAttributes: ISoundAttributes;
@@ -85,6 +84,8 @@ export declare class PlayerCore {
     protected _loadSoundUsingAudioElement(sound: ISound): Promise<void>;
     protected _loadSoundUsingRequest(sound: ISound): Promise<void>;
     protected _decodeSound({ sound }: IDecodeSoundOptions): Promise<void>;
+    manuallyUnlockAudio(): Promise<void>;
+    checkIfAudioIsUnlocked(): Promise<boolean>;
     play({ whichSound, playTimeOffset }?: IPlayOptions): Promise<ISound>;
     protected _play(sound: ISound): Promise<void>;
     protected _playAudioBuffer(sound: ISound): Promise<void>;

dist/index.min.js

@@ -52,6 +52,7 @@ export class PlayerAudio {
     protected _audioElement: HTMLAudioElement = null;
     protected _mediaElementAudioSourceNode: MediaElementAudioSourceNode = null;
     protected _isAudioUnlocked: boolean = false;
+    protected _isAudioUnlocking: boolean = false;
 
     constructor(options: IAudioOptions) {
 
@@ -114,19 +115,23 @@ export class PlayerAudio {
     protected _addFirstUserInteractionEventListeners(): void {
 
         if (this._options.createAudioContextOnFirstUserInteraction) {
-            document.addEventListener('touchstart', this.unlockAudio.bind(this));
-            document.addEventListener('touchend', this.unlockAudio.bind(this));
+            document.addEventListener('keydown', this.unlockAudio.bind(this));
             document.addEventListener('mousedown', this.unlockAudio.bind(this));
+            document.addEventListener('pointerdown', this.unlockAudio.bind(this));
+            document.addEventListener('pointerup', this.unlockAudio.bind(this));
+            document.addEventListener('touchend', this.unlockAudio.bind(this));
         }
 
     }
 
     protected _removeFirstUserInteractionEventListeners(): void {
 
         if (this._options.createAudioContextOnFirstUserInteraction) {
-            document.removeEventListener('touchstart', this.unlockAudio.bind(this));
-            document.removeEventListener('touchend', this.unlockAudio.bind(this));
+            document.removeEventListener('keydown', this.unlockAudio.bind(this));
             document.removeEventListener('mousedown', this.unlockAudio.bind(this));
+            document.removeEventListener('pointerdown', this.unlockAudio.bind(this));
+            document.removeEventListener('pointerup', this.unlockAudio.bind(this));
+            document.removeEventListener('touchend', this.unlockAudio.bind(this));
         }
 
     }
@@ -135,10 +140,17 @@ export class PlayerAudio {
 
         return new Promise((resolve, reject) => {
 
-            if (this._isAudioUnlocked) {
+            if (this._isAudioUnlocked || this._isAudioUnlocking) {
+                return resolve();
+            }
+
+            // https://webkit.org/blog/13862/the-user-activation-api/
+            if (typeof navigator.userActivation !== 'undefined' && navigator.userActivation.isActive) {
                 return resolve();
             }
 
+            this._isAudioUnlocking = true;
+
             // make sure the audio context is not suspended
             // on android this is what unlocks audio
             this.getAudioContext().then(() => {
@@ -166,28 +178,43 @@ export class PlayerAudio {
                         // as a direct result of an user interaction
                         // after it got unlocked we re-use that element for all sounds
                         this._createAudioElementAndSource().then(() => {
-
                             this._isAudioUnlocked = true;
+                            this._isAudioUnlocking = false;
                             return resolve();
-                        }).catch(reject);
+                        }).catch((error) => {
+                            console.error(error);
+                            this._isAudioUnlocking = false;
+                            return reject();
+                        });
 
                     } else if (this._options.loadPlayerMode === 'player_mode_ajax') {
                         this._isAudioUnlocked = true;
+                        this._isAudioUnlocking = false;
                         return resolve();
                     }
 
                 };
 
                 bufferSource.buffer = placeholderBuffer;
                 bufferSource.connect(this._audioContext.destination);
+                // attempt to play the empty buffer to check if there is an error
+                // or if it can be played, in which case audio is unlocked
                 bufferSource.start(0);
 
-            }).catch(reject);
+            }).catch((error) => {
+                console.error(error);
+                this._isAudioUnlocking = false;
+                return reject();
+            });
 
         });
 
     }
 
+    public isAudioUnlocked() {
+        return (this._isAudioUnlocked || (typeof navigator.userActivation !== 'undefined' && navigator.userActivation.isActive)) ? true : false;
+    }
+
     protected async _createAudioElementAndSource(): Promise<void> {
 
         await this._createAudioElement();
@@ -447,6 +474,8 @@ export class PlayerAudio {
 
         } else if (this._options.loadPlayerMode === 'player_mode_audio') {
 
+            await this._createAudioElementAndSource();
+
             // create the sound gain node
             sound.gainNode = this._mediaElementAudioSourceNode.context.createGain();