View source for Implementing the Mouse Lock API in Firefox

==Introduction==

This is a working document for the implementation of the [http://dvcs.w3.org/hg/webevents/raw-file/default/mouse-lock.html Mouse Lock API spec] in Mozilla by students in David Humphrey's Mozilla Development class at Seneca College.

Please add, edit, correct, expand, etc. as necessary.  This page should contain any links or other info we need.

==Participants==

While the project is primarily meant for students in DPS909/OSD600, feel free to join us if you want to work on things.

* David Humphrey (lead developer, professor, [http://twitter.com/humphd @humphd])
* Hasan Kamal-Al-Deen (tardy student, [http://twitter.com/NorthWind87 @NorthWind87])
* Matthew Schranz (Student, OSD600, [http://twitter.com/mjschranz @mjschranz])
* Yevgeniy Ivanchenko (Student, OSD600)
* Chris Gosselin (Student, OSD600)
* Anurag Bhatnagar (Student, DPS909, [http://twitter.com/anuragbh @anuragbh])
* Raymond Hung (Wannabe Developer, Student, [http://twitter.com/Raymond_Hung @Raymond_Hung])
* Ausley Johnson(Student, OSD600)
* Jesse Silver (Student, OSD600)
* Ching Wei Tseng(Student, DPS909)
* Michelle Mendoza (Student, DPS909)
* Archana Sahota (Student, DPS909)
* Greg Krilov (Student, DPS909)
* Roman Hotin (Student, DPS909)
* Sergiu Ecob (Student, OSD600)
* Jordan Raffoul (Student)
* Hyungryul Chun (Student, DPS909)
* James Boelen ([http://25.media.tumblr.com/tumblr_ln817mfAfn1qlj6hwo1_500.jpg Masked Crusader], [http://twitter.com/jamesboelen @jamesboelen])
* Jacky Siu (Student, OSD600)
* Abhishek Bhatnagar (Student, [http://twitter.com/abhishekToronto @abhishekToronto])
* Diogo Golovanevsky Monteiro ([http://twitter.com/diogogmt @diogogmt])
* Simon de Almeida(Student) ([http://twitter.com/simon661 @simon661])
* Stanley Tsang (Student, DPS909)
* Denise Rigato (Student, DPS909)
* Qian (Ken) Xu (Student, DPS909)
* Moussa Tabcharani (Student, DPS909)
* Keyan Ren (Student, OSD600)
* Joseph Hughes (Student, OSD600)

==Communication==

Development work will be done using a combination of the following:

* In class discussions: 11:40-1:30 Tuesday and Thursday in TEL T2110
* IRC discussions: [irc://irc.mozilla.org/seneca #seneca], [irc://irc.mozilla.org/paladin #paladin]
* Questions: ask and answer any questions here: [[Mouse Lock Implementation FAQ]]
* Blog Series: http://vocamus.net/dave/?cat=28

==Tasks==

===Getting Started===

# Clone our repo and build a debug version locally
# Get a https://bugzilla.mozilla.org account and CC yourself on the [https://bugzilla.mozilla.org/show_bug.cgi?id=633602 bug].
# Set a Watch on this page and the Q/A page so you know when things change.
# Break the spec down into an itemized list of things we need to do, tests we need to write, features we have to add, edge cases we have to worry about, demos we need to build, etc.  Put the info into this page.  We need to know everything we'll have to write and schedule when we'll do each bit.
# Blog about your work on this implementation
# Add questions/answers to [[Mouse Lock Implementation FAQ]]

===High-Level Mouse Lock Implementation Tasks===

====Implementation====

# <s>No mouse cursor is displayed when the mouse is locked</s> - rhung
# <s>MouseLockable DOM Implementation, navigator.pointer ([[Notes on MouseLock DOM Implementation Nov 13, 2011]])</s> - humph
## <strike>[[void lock (in Element target, optional in VoidCallback successCallback, optional in VoidCallback failureCallback);]]</strike> diogogmt, humph
## <strike>void unlock ();</strike> humph, diogogmt
## <strike>bool islocked ();</strike> - humph
# <strike>Mouse Lock Platform Implementations</strike> JSilver999, humph
## <strike>Windows: [http://msdn.microsoft.com/en-us/library/windows/desktop/ms648394%28v=vs.85%29.aspx SetCursorPos(x, y)]</strike>
## <strike>OS X [http://www.cocoabuilder.com/archive/cocoa/77124-setting-cursor-position.html CGWarpMouseCursorPosition(CGPointMake(x, y))]</strike>
## <strike>Linux (GTK) [http://www.gtkforums.com/viewtopic.php?t=182 gdk_display_warp_pointer (display, screen, x, y)], add to http://mxr.mozilla.org/mozilla-central/source/widget/src/gtk2/nsWindow.cpp ???</strike>
## Mobile?
# <strike>Investigate whether we can get movement information directly from OS</strke> - northWind
# <s>mouselocklost event DOM Implementation</s> - diogogmt
# <s>Extend MouseEvent DOM implementation with movementX, movementY</s> - humph, JSilver999
## <s>[[Stack Trace for nsDOMMouseEvent::nsDOMMouseEvent ctor]] (created on mouse move)</s>
## <s>[[Stack Trace for nsDOMMouseEvent::GetScreenX]]</s>
## <s>[[Where to store the state info (e.g., previous position) between mouse events?]]</s>
# <s>The browser must exit the mouse lock state if the user agent, window, or tab loses focus</s> - diogogmt
# <strike>Fixup NULLs being returned from lock() C++ implementation, should be NS*</strike> - Anachid
# <strike>Mouse lock should only work when in Full Screen Mode</strike> - diogogmt, rhung
# <strike>[[Refactor nsIDOMNavigator changes for pointer attribute to be in separate interface]]</strike> humph
# <strike>Do we need to do conditional compilation for mouse lock?</strike> humph (not going to bother for now)
# <strike>When mouse lock is enabled clientX, clientY, screenX, and screenY must hold constant values as if the mouse did not move at all once mouse lock was entered.</strike> humph
# <s>Freeze mouse pointer in centre of window when mouse lock is enabled (e.g., moving the mouse causes an event, but forces the mouse to go back to the original position).</s> - JSilver999
# <strike>"Events that require the concept of a mouse cursor must not be dispatched (for example: mouseover, mouseout)"</strike> - humph
# <strike>Figure out [[Mac Crash with Jesse's SynthesizeMouseMove change]]</strike> humph
# <strike>When the locked element is removed from the DOM Tree, the mouse should be unlocked</strike> diogogmt
# <strike>Save the screenX and screenY position before locking the mouse.</strike> - humph
# <strike>Reset the mouse position back to the original position when unlocking.</strike> - humph
# <s>"When unlocked, the system cursor can exit and re-enter the user agent window. If it does so and the user agent was not the target of operating system mouse move events then the most recent mouse position will be unknown to the user agent and movementX/Y can not be computed and must be set to zero"</s> diogogmt
# <s>Trying to lock a locked element should fire the success callback</s> CloudScorpion
# <s>Before locking the mouse check if the element is a DOM element and if it is in the DOM Tree</s> diogogmt
# <s>Fix accurateness of mouse positioning on unlock() (should be the same point as when lock() was called).  Currently works, but is offset.  See nsEventStateManager::SetMouseLock. </s> JSilver999
# <s>Restructure Lock method to do most of its operations in a separate thread.</s> humph
# <s>Fix license headers for new files to use proper [http://www.mozilla.org/MPL/boilerplate-1.1/ MPL boilerplate]</s> humph
# <s>Do we need to add a user pref to enable/disable mouse lock?  Nice to have, not blocking.</s> northwind, mjschranz
# "Once mouse lock is acquired, stop mouse events from being fired to other elements that are not locked (e.g., only fire to locked element)."  Only the fullscreen element will get events.  Need advice in review on how to do this properly.

=====Out of Scope Implementation=====

Because we are only implementing Mouse Lock for Fullscreen elements, some aspects of the spec can/must be put off until later.  Other items below are simply not in scope for this first round of implementation.

# The ESC key should exit mouse lock.  This will currently exit fullscreen, and therefore mouse lock - diogogmt
# "User agents may prompt for confirmation before locking, this preference may be saved as a content setting"  How to deal with this?  What UI do we use?  See also, "Repeated escapes of mouse lock can signal user agent to not re-lock the mouse without more specific user intent gesture, e.g. similar to how Chrome suppresses repeated alert() calls"
# "The Mouse Lock API must exit the mouse lock state if the user agent, window, or tab loses focus"
# Clip the mouse so it doesn't leave the locked element with a mouse movement large enough to exceed its bounds.  See:
## [http://msdn.microsoft.com/en-us/library/windows/desktop/ms648383%28v=vs.85%29.aspx ClipCursor] on Windows
## [http://developer.apple.com/library/mac/#documentation/GraphicsImaging/Reference/Quartz_Services_Ref/Reference/reference.html CGAssociateMouseAndMouseCursorPosition and CGGetLastMouseDelta] on OS X
## [http://www.x.org/archive/X11R6.8.2/doc/XGrabPointer.3.html XGrabPointer] on Linux

===Tests===

The central repo with all the tests are located [https://github.com/rhung/mozilla-central/tree/mouselock-tests here]. For more information on how to make mochitests and how to send the pull request to the appropriate person, check out the [http://zenit.senecac.on.ca/wiki/index.php/Mochitest_FAQ Mochitest FAQ] List tests we need below:
# <s>"there is no limit to how far movement can go...not limited by screen boundaries" -- mouse lock should mean infinite movement in the X and Y axes...There will be no limit to movementX/Y values if the mouse is continuously moved in a single direction"</s>  - '''hchun'''
# <s>"The concept of the mouse cursor will have been removed, and it will not move off the window or be clamped by a screen edge"</s> - '''hchun'''
# "no mouse cursor is displayed" -- mouse cursor should be hidden while locked - '''dvillase'''
# <s>navigator.pointer (readonly) is a MouseLockable</s> -  <strong>abhatnagar</strong>
# <s>MouseLockable has lock(), unlock(), islocked()</s> -  <strong>abhatnagar</strong>
# <s>islocked() returns true if mouse is locked, false if not locked</s> - <strong>JSilver999</strong>
# <s>lock(target) expects a DOM element, and takes two optional callbacks: successcallback, failurecallback.</s> - <strong>JSilver999</strong>
# <s>lock() should return immediately and call callbacks when lock succeeds or fails</s> - <strong>JSilver999</strong>
# <s>"Mouse lock must succeed only if the window is in focus" - '''johnno'''</s>
# <s>"Mouse lock must succeed only if...the user-agent is the active application of the operating system"</s> - '''johnno'''
# <s>"The target of lock need not be in focus" - '''johnno'''</s>
# <s>"Mouse lock must succeed only if the target is in the DOM tree"</s> - '''Anachid'''
# <s>"If the target is removed from the DOM tree after mouse lock is entered then mouse lock will be lost."</s> - '''Anachid'''
# <s>"If the mouse is already locked to the same element, a repeated call to lock will succeed and the successCallback called"</s> - '''jboelen'''
# <s>"If another element is locked [and lock() is called] a user agent must transfer the mouse lock to the new target and call the mouselocklost callback for the previous target"</s> - '''jboelen'''
# <s>"The Mouse Lock API must provide a default system action to unlock the mouse" namely ESC.</s> - '''CloudScorpian'''
# <s>"Once in the locked state the user agent must fire all relevant user generated MouseEvent events (for example: mousemove, mousedown, mouseup, click, wheel)[DOM-LEVEL-3-CORE] to the target of mouse lock, and not fire mouse events to other elements"</s> - '''rhung'''
# <s>"Events that require the concept of a mouse cursor must not be dispatched (for example: mouseover, mouseout)"</s> - '''rhung'''
# "Movement and button presses of the mouse must not cause the window to lose focus" - 
# <s>"Synthetic mouse events created by application script act the same regardless of lock state"</s> - '''Tentacle'''
# <s>"The unlock method cancels the mouse lock state"</s> - '''abhatnagar1'''
# "[Upon unlock() t]he system mouse cursor must be displayed again and positioned at the same location that it was when mouse lock was entered (the same location that is reported in screenX/Y when the mouse is locked)" - '''dvillase'''
# <s>"When mouse lock is lost or disabled for any reason user agents must fire an event named mouselocklost with its bubble attribute set to true to the mouse lock target element"</s> - '''stsang'''
# <s>MouseEvent must contain (readonly) movementX and movementY</s> - '''KeyR, JSilver999'''
# <s>"The members movementX and movementY must provide the change in position of the mouse, as if the values of screenX/Y were stored between two subsequent mousemove events eNow and ePrevious and the difference taken movementX = eNow.screenX-ePrevious.screenX"</s> - '''KeyR, JSilver999'''
# <s>"movementX/Y must be valid regardless of mouse lock state"</s> - '''KeyR, JSilver999'''
# <s>"When unlocked, the system cursor can exit and re-enter the user agent window. If it does so and the user agent was not the target of operating system mouse move events then the most recent mouse position will be unknown to the user agent and movementX/Y can not be computed and must be set to zero"</s> - '''moussa1'''
# <s>"When mouse lock is enabled clientX, clientY, screenX, and screenY must hold constant values as if the mouse did not move at all once mouse lock was entered"</s> - '''jbraffoul'''
# <s>"The Mouse Lock API must exit the mouse lock state if the user agent, window, or tab loses focus"</s> - '''drigato'''
# Test to make sure that mouse lock only occurs when an element is in full screen mode (not F11 or done via the menus). This includes:
## Switching focus to another window  - '''jsiu3'''
# <s>Tests for mouselocklost event</s> - '''stsang'''

=====Reviewing Tests=====

There are a series of common mistakes in the tests that need to get fixed.  Here are some of them:

# Need to clean-up tabs vs. spaces and indentation issues (2-spaces per tab) in many test files.  Use https://github.com/einars/js-beautify.  [https://github.com/rhung/mozilla-central/blob/mouselock-tests/dom/tests/mochitest/mouselock/test_userPref.html This is a test that is formatted correctly], in terms of indentation and spaces vs. tabs.
# No line of code should be 80 characters or longer--break them so they are under 80
# No Windows end-of-lines, use Unix end-of-lines
# No whitespace at the end of a line
# Run your test code through http://www.jshint.com/. Note, it will complain about unknown globals like SimpleTest.  Make sure the rest of the JavaScript is good.
# Remove unnecessary comments.  Only things that explain the test.
# If you need constants, use const instead of var
# Don't use variables if they aren't needed.  For example, don't introduce a variable to store a value, only to pass it into ok() or is().  Just test the expression in ok() or is().
# Make sure your JavaScript follows proper naming: goodVariableName, bad_variable_name, badvariablename;
# Tests should follow the template as closely as makes sense: http://pastebin.com/vxmsepVh
# Tests should include a simple comment block describing what is being tested
# Only use <code>SimpleTest.waitForFocus()</code> if you really need it.
# Be consistent with "..." vs. '...' for strings.  Pick one and use it throughout the file.
# Remove <code>console.log()</code>
# Prefer <code>document.body</code> to <code>document.getElementsByTagName('body')[0]</code>
# Braces on the same line: <code>if (...) {\n</code> and <code>} else {\n</code>
# An error message for is() or ok() of "Error message" is not acceptable.  Make sure you understand the failure.
# If you need to actually lock the mouse, you'll need <code>SpecialPowers.setBoolPref("full-screen-api.allow-trusted-requests-only", false);</code> to allow non-user initiated fullscreen mode (i.e., normally it requires a user to click a button or trigger some other event).
# If you're testing that variable foo or expression fooFunction() are true, use ok(), not is(): <code>ok(foo, "Error message if not true.");</code> or <code>ok(fooFunction(), "Error message if function returns false");</code>.  Don't do <code>is(foo, true, "Error message if false.");
# Make sure code that relies on things happening in asynchronous code gets called in a callback or event handler, not on the next line.  For example, if you call navigator.pointer.lock(), you can't check navigator.pointer.islocked() on the next line, you need to use the successCallback/errorCallback.  Same for focus or blur calls and events.  Actions that trigger something happening in the future often need a callback or event handler.
# Prefer <code>i++;</code> to <code>i+=1;</code>

One or more tests have to deal with switching the lock between multiple elements, and the [[Changes to Fullscreen Unlock|spec is changing]] on that front.

====Test Reviews====

The following tests need review and/or fixes, and to then get updated in [https://github.com/rhung/mozilla-central/tree/mouselock-tests/dom/tests/mochitest/mouselock rhung's mouslock-tests] branch on github.  Please add your name beside test(s) you are reviewing/fixing.  Once a test is complete and updated in his branch, please cross it off below.
{| border=1 style="border: 1px solid darkgray;"
| Num
| Name
| Owner
| Has Harness
| Rev.
| Peer Rev.
| Pull
| Contrib.
|- 
| colspan=8 bgcolor="yellow" style="text-align:center;" | '''Ensure you have actually tested your test before sending a pull request'''
|-
|- 
| colspan=8 bgcolor="lightgreen" style="text-align:center;" | '''Fill Out the Contrib cell with "First Last <email>" or leave blank for none.'''
|-
|1
|test_FullScreenHarness.html
|Anachid
|X
|
|humph
|
|Ching Wei Tseng (Steven) <steven_tseng15@hotmail.com>
|-
|2
|mouselock_util.js
|Anachid
|X
|
|humph
|
|Ching Wei Tseng (Steven) <steven_tseng15@hotmail.com>
|-
|3
|file_DOMtree.html
|Anachid
|X
|
|humph
|
| Ching Wei Tseng (Steven) <steven_tseng15@hotmail.com>
|-
|4
|test_MouseEvents.html
|rhung
|
|
|
|
|
|-
|5
|test_TargetOutOfFocus.html
|johnno
|
|
|
|
|
|-
|6
|test_UserAgentIsActive
|johnno
|
|
|
|
|
|-
|7
|test_constantxy.html
|jbraffoul
|
|
|
|
|
|-
|8
|test_defaultUnlock.html
|KeyR
|
|X
|nm486
|
|
|-
|9
|test_doubleLockCallBack.html
|jboelen
|
|X
|rhung
|
|
|-
|10
|test_exitMouselockOnLoseFocus.html
|drigato / johnno
|
|
|
|
|
|-
|11
|test_isInstanceofMouselockable.html
|abhatnagar
|
|X
|
|
|
|-
|12
|test_limitlessScroll.html
|hchun
|
|X
|
|
|
|-
|13
|test_mouseLockableHasRequiredMethods.html 
|abhatnagar
|
|X
|
|
|
|-
|14
|test_mousePos.html
|jboelen
|
|
|
|
|
|-
|15
|test_mouselocklost.html
|nm486
|
|
|
|
|
|-
|16
|test_movement.html
|hchun
|
|
|
|
|
|-
|17
|test_navigatorPointer.html
|abhatnagar
|
|
|
|
|
|-
|18
|test_syntheticMouseEvent.html
|
|
|
|
|
|
|-
|19
|test_userPref.html
|mjschranz
|
|
|
|
|
|-
|20
|test_fullscreen.html
|abhatnagar1
|x
|x
|
|
|Abhishek Bhatnagar <abhatnagar1@learn.senecac.on.ca>
|-
|21
|test_syntheticMouseEvent.html
|Tentacle
|x
|x
|x
|x
|Qian Xu <qxu26@learn.senecac.on.ca>
|}


# test_FullScreenHarness.html - Anachid
# mouselock_util.js - Anachid
# file_DOMtree.html - Anachid
# <s>test_LockWhenOutOfFocus.html</s> Note: Removed
# test_MouseEvents.html - rhung
# <s>test_TargetOutOfFocus.html</s> - johnno
# test_UserAgentIsActive.html - johnno
# test_constantxy.html - jbraffoul
# <s>test_defaultUnlock.html</s>
# <s>test_doesunlock.html</s> Note: Removed, redundant;
# <s>test_doubleLockCallBack.html</s>
# <s>test_exitMouselockOnLoseFocus.html</s> drigato,johnno(merged)
# <s>test_fullscreen.html</s>
# <s>test_isInstanceofMouselockable.html</s> - abhatnagar
# <s>test_limitlessScroll.html</s> - hchun
# <s>test_lockLostCallBack.html</s>
# <s>test_mouseLockableHasRequiredMethods.html</s> - abhatnagar
# test_mousePos.html
# <s>test_mouselocklost.html</s> - nm486
# <s>test_movement.html</s> -hchun
# <s>test_navigatorPointer.html</s> - abhatnagar
# <s>test_syntheticMouseEvent.html</s>
# <s>test_userPref.html</s> - mjschranz

====Using the Test Harness====
'''Note: This harness assumes that your test runs properly as a stand-alone application and is not being used as one of the test amongst others.'''

The problem with the test timing out is due to the fact that the Mochitest '''iframe does not allow fullscreen''' from the html code inside the frame. To solve this problem, it needs a small work around like using the harness.

To make use of the harness, you will need to add the '''mouselock_util.js''' file to your test html file. I would recommend renaming your file to '''file_something.html''' (vs. test_something.html), as this denotes that this not a direct test under the mochitest.

Thus, add the following line, AFTER the EventUtil.js and SimpleTest.js script tag

<pre>
<script type="application/javascript" src="mouselock_util.js"></script>
</pre>

Another, thing that needs to be done is to add your file to the '''gTestFiles''' array inside the '''test_FullScreenHarness.html''' file

After that, do a run of the whole mochitest directory on your machine before doing a pull request.
(use this):
<pre>
TEST_PATH=dom/tests/mochitest/mouselock/ make -C $(replace with OBJDIR) mochitest-plain
</pre>


Finally, since the fullscreen needs the harness, the harness will provide the special powers. So please no '''NOT add any special powers''' in your child files.

====Demos, Docs, Other====

# Proper IDL documentation for navigator.pointer (see example in https://mxr.mozilla.org/mozilla-central/source/dom/interfaces/html/nsIDOMHTMLElement.idl#103), MouseLockable and its methods, MouseLockLost event, etc.
# Write a JavaScript library to somehow combine element.mozRequestFullScreen() and navigator.pointer.lock().  It would be good to hide the complexities of doing fullscreen then locking in a single API call. 

<pre>
var canvas = document.getElementById('canvas');

// Will put element into fullscreen then try to do mouse lock
var lock = acquireMouseLock({
  element: canvas,
  onLock: function() {
    // OPTIONAL
    // fullscreen + mouse lock are now setup for element
  },
  onUnlock: function() {
    // OPTIONAL
    // the user or browser took us out of fullscreen/mouse lock
  },
  onError: function() {
    // OPTIONAL
    // there was an error getting fullscreen/mouse lock
  },
  onMovemennt: function(e) {
    // OPTIONAL
    // movementX and movementY for mousemove are passed in
    var deltaX = e.movementX;
    var deltaY = e.movementY;
  }
});

...
lock.islocked; // true or false
lock.unlock();
</pre>

# Mouse lock specification fix requests
# Convert Rescue Fox to use Mouse Lock, see https://github.com/mozilla/rescuefox
# <strike>Convert http://cjcliffe.github.com/CubicVR.js/cubicvr/samples/fps_demo/level1.html to use Mouse Lock</strike> - JSilver999
# Create a tutorial on how to use Mouse Lock, with code examples
# Add demo pages to [http://humphd.github.com/mozilla-central/mouselock/index.html gh-pages branch]
# Review https://developer.mozilla.org/en/API/Mouse_Lock_API for correctness with spec + our implementation

==Resources==

* Spec Document: http://dvcs.w3.org/hg/webevents/raw-file/default/mouse-lock.html
* Implementation Repo: https://github.com/humphd/mozilla-central on the '''mouselock''' branch.
* Mozilla Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=633602
* WebKit Bug: https://bugs.webkit.org/show_bug.cgi?id=68468
* Chromium Bug: http://code.google.com/p/chromium/issues/detail?id=72754
* Points of Interest: http://zenit.senecac.on.ca/wiki/index.php/Mouse_Lock_API_Points_of_Interest
* Mochitest: https://developer.mozilla.org/en/Mochitest
* Mochitest FAQ: http://zenit.senecac.on.ca/wiki/index.php/Mochitest_FAQ
* Mozilla Cross Reference: http://mxr.mozilla.org/mozilla-central/
* MouseLock Demos Link: http://humphd.github.com/mozilla-central/mouselock/index.html
* MouseLock Stage Build download: http://stage.mozilla.org/pub/mozilla.org/firefox/try-builds/david.humphrey@senecac.on.ca-f4e5849f1ae7/