1
edit
Changes
→Review Issues
===High-Level Mouse Lock Implementation Tasks===
====Implementation====
# <strike>Mouse lock should only work when in Full Screen Mode</strike> - diogogmt, rhung
# <strike>[[Refactor nsIDOMNavigator changes for pointer attribute to be in separate interface]]</strike> humph
# <strike>Do we need to do conditional compilation for mouse lock?</strike> humph (not going to bother for now)
# <strike>When mouse lock is enabled clientX, clientY, screenX, and screenY must hold constant values as if the mouse did not move at all once mouse lock was entered.</strike> humph
# <s>Freeze mouse pointer in centre of window when mouse lock is enabled (e.g., moving the mouse causes an event, but forces the mouse to go back to the original position).</s> - JSilver999
# <strike>"Events that require the concept of a mouse cursor must not be dispatched (for example: mouseover, mouseout)"</strike> - humph
# <strike>Figure out [[Mac Crash with Jesse's SynthesizeMouseMove change]]</strike> humph
# <strike>When the locked element is removed from the DOM Tree, the mouse should be unlocked</strike> diogogmt
# <strike>Save the screenX and screenY position before locking the mouse.</strike> - humph
# <strike>Reset the mouse position back to the original position when unlocking.</strike> - humph
# <s>"When unlocked, the system cursor can exit and re-enter the user agent window. If it does so and the user agent was not the target of operating system mouse move events then the most recent mouse position will be unknown to the user agent and movementX/Y can not be computed and must be set to zero"</s> diogogmt# <s>Trying to lock a locked element should fire the success callback</s> CloudScorpion# <s>Before locking the mouse check if the element is a DOM element and if it is in the DOM Tree</s> diogogmt# <s>Fix accurateness of mouse positioning on unlock() (should be the same point as when lock() was called). Currently works, but is offset. See nsEventStateManager::SetMouseLock.</s> JSilver999# <s>Restructure Lock method to do most of its operations in a separate thread. diogogmt</s> humph# <s>Fix license headers for new files to use proper [http://www.mozilla.org/MPL/boilerplate-1.1/ MPL boilerplate]</s> humph# <s>Do we need to add a user pref to enable/disable mouse lock? Nice to have, not blocking.</s> northwind, mjschranz# "Once mouse lock is acquired, stop mouse events from being fired to other elements that are not locked (e.g., only fire to locked element)." Only the fullscreen element will get events. Need advice in review on how to do this properly.
=====Out of Scope Implementation=====
Because we are only implementing Mouse Lock for Fullscreen elements, some aspects of the spec can/must be put off until later. Other items below are simply not in scope for this first round of implementation.
# The ESC key should exit mouse lock. This will currently exit fullscreen, and therefore mouse lock - diogogmt
# "Once User agents may prompt for confirmation before locking, this preference may be saved as a content setting" How to deal with this? What UI do we use? See also, "Repeated escapes of mouse lock is acquired, stop mouse events from being fired can signal user agent to other elements that are not locked (re-lock the mouse without more specific user intent gesture, e.g.similar to how Chrome suppresses repeated alert() calls"# "The Mouse Lock API must exit the mouse lock state if the user agent, window, only fire to or tab loses focus"# Clip the mouse so it doesn't leave the locked elementwith a mouse movement large enough to exceed its bounds. See:## [http://msdn.microsoft.com/en-us/library/windows/desktop/ms648383%28v=vs.85%29.aspx ClipCursor] on Windows## [http://developer.apple.com/library/mac/#documentation/GraphicsImaging/Reference/Quartz_Services_Ref/Reference/reference.html CGAssociateMouseAndMouseCursorPosition and CGGetLastMouseDelta] on OS X## [http://www.x.org/archive/X11R6.8.2/doc/XGrabPointer.3.html XGrabPointer] on Linux ====Review Issues==== * <s>Rewrite DOM node removal unlocking logic to use nsIMutatationObserver</s> humph, diogogmt* <s>Space, bracing, tabs, etc style nits</s> humph* <s>clientX, clientY, screenX, screenY values should follow spec as per [https://bugzilla.mozilla.org/show_bug.cgi?id=633602#c55 scheib's comment]</s> diogogmt* <s>Confirm |aEvent->lastRefPoint = nsIntPoint(bounds.width/2, bounds.height/2);| that / 2 is right</s> diogogmt* <s>"nsCOMPtr<nsIContent> mMouseLockedElement; - Shouldn't this be a static member variable? Or how does the patch handle cases when the document which has iframe and mouse moves over those iframes." Only Probably wants a test for this case, too.</s> diogogmt* <s>"nsDOMMouseLockable::ShouldLock...QI to nsINode and check IsInDoc()"</s> diogogmt* <s>"nsRefPtr<nsMouseLockableRequest> request = new nsMouseLockableRequest(aSuccessCallback, aFailureCallback); -- You should store aTarget (QI'ed to nsINode or nsIContent or Element or nsIDOMEventTarget) so that when calling callback you push Cx to stack; nsCxPusher pusher: // defined in nsContentUtils.h NS_ENSURE_STATE(pusher.Push(target));"</s> - humph* <s>"nsDOMMouseLockable looks like it should be cycle collectable"</s> humph* <s>Break .pointer out of Navigator, and add nsIDOMMozNavigatorPointer (not sure if this name is best) as a new interface to Navigator, similar to nsIDOMMozNavigatorBattery.</s> - humph, diogogmt* <s>Rename for moz* prefix. The API should be probably prefixed. So, navigator.mozPointer and all the interfaces should start with nsIDOMMoz. Similar to what webkit* is doing:</s> - humph** <s>navigator.webkitPointer.unlock();</s>** <s>navigator.webkitPointer.lock();</s>** <s>navigator.webkitPointer.isLocked();</s>** <s>document.body.addEventListener("webkitpointerlocklost", ...</s>** <s>mouseMoveEvent.webkitMovementX</s>** <s>mouseMoveEvent.webkitMovementY</s>* Follow discussion unfolding here for other changes: http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/1558.html* Finish tests, updated to changes above.* <s>Update patch to trunk</s> - humph* Figure out why DOMMouseScroll events aren't being retargeted to the fullscreen locked element will get events.
The central repo with all the tests are located [https://github.com/rhung/mozilla-central/tree/mouselock-tests here]. For more information on how to make mochitests and how to send the pull request to the appropriate person, check out the [http://zenit.senecac.on.ca/wiki/index.php/Mochitest_FAQ Mochitest FAQ] List tests we need below:
New Tests (not sure if it will work on mochitest, and the test will possibly fail as dave is working on this patch)# When there is a parent element with multiple child element, have a test to check that all mouse events fired inside the child element are suppressed and is captured by the parent element. (this may also include the scenario where there is an iframe in the parent element)# A Test to check that repeated move of the mouse to a certain direction that will exceed the screen size does not break the mouselock (this may conflict with file_movementXY.html and we may need to add more test cases) ====Changes To Be Applied To Submitted Tests [Deprecated]==== The following table is used to maintain a small note on the task for each test. It will include note from the review by Mozilla and issue we overlooked. {| border=1 style="border: 1px solid darkgray;"| Num| Test Name| Reviewer| Notes|- | colspan=4 bgcolor="yellow" style="text-align:center;" | '''Please add your name to the section you want to review and fix'''|-|- | colspan=8 bgcolor="lightgreen" style="text-align:center;" | '''Fill out the Notes section for any issues that are outstanding'''|-|1|test_DOMtree.html||* northWind Comments:** Should Delete the element from DOM without exiting fullscreen and then check for pointer.lock instead of the current flow.|-|2|test_MouseEvents.html||* rhung Comments:** Should move the tests that require mouselock into the success callback of the lock function.** Figure out why mouseleave and mouseout tests don't work for Macs.|-|3||||-|} ====Remaining Test Reviews [Deprecated]====The following tests need investigation. Please add your name beside each test(s) you are assessing/reviewing/fixing/peer reviewing.Once the cause for rejection is clear, indicate it by placing an X in any of the columns and optionally filling out the comments column. Indicate the action taken to resolve the issue in the action taken column. Example actions include:# '''DELETED'''# '''FIXED''' Other actions are also acceptable where appropriate.Once fixed, please pull request to rhung's [https://github.com/rhung/mozilla-central/tree/mouselock-tests/dom/tests/mochitest/mouselockrhung's mouselock-tests] branch and indicate that you've done so in the pull requested column. Ensure that you add '''[POST-SUBMIT]''' to the name of your pull request. {| border=1 style="border: 1px solid darkgray;"| Num| Name| Primary Reviewer| Timing issues?| Needs Harness?| Needs focus?| Formatting issues?| Other? (Indicate in comments)| Peer Reviewers| Peer reviewed?| Action Taken| Pull requested?| bgcolor="red" style="text-align:center;" | Updated with Patchv3 | Comments|- | colspan=14 bgcolor="yellow" style="text-align:center;" | '''Ensure you have actually tested your test before sending a pull request'''|-|-|<s>106</s>|<s>test_UserAgentIsActive</s>|<s>diogogmt</s>||<s>X</s>||<s>X</s>|<s>X</s>|<s>northwind</s>||<s><b>IMPOSSIBLE IN MOCHITEST</b></s>|||<s>* northWind Comments prior to diogo re-writing user agent test:** navigator.userAgent does not indicate focus status of browser.** test should be testing for full screen if user agent is in focus then testing that full screen is unavailable when user agent is not in focus.** test uses tabs, should use spaces.</s>|-|<s>109</s>|<s>file_doubleLockCallBack.html</s>|<s>northWind</s>||X|X|X||<s>rhung</s>|X|<s><b>FIXED, PEER REVIEWED, AND ADDED TO REPOSITORY</b></s>|X||* <s>For Peer Reviewer: https://github.com/rhung/mozilla-central/blob/mouselock-tests/dom/tests/mochitest/mouselock/file_doubleLockCallBack.html</s>|-|<s>110</s>|<s>test_exitMouselockOnLoseFocus.html</s>|<s>diogogmt</s>||<s>X</s>|<s>X</s>|<s>X</s>||<s>northWind</s>|<s>X</s>|<s><b>FIXED, PEER REVIEWED, AND ADDED TO REPOSITORY</b></s>||||-|114|test_mousePos.html|diogogmt|||||||||||A summary of the test: *Move the mouse outside the browser boundaries. *Move the mouse back inside the browser. *Checks if movementXY were set to zero, since the last refPoint is unknownThe problem is that synthesizeMouse doesn't move the mouse outside the boundaries of the specified element. If the X/Y coords passed to synthesizeMouse are bigger than the X/Y of the element receiving the event, it won't move the cursor. I'm logging all the screen/client/movementXY values, and if the coords passed to synthesizeMouse are bigger than the boundaries of the element receiving the mousemove, the test will break, since the mouse is never moved. [http://pastebin.com/MhP62BtC code] |-|<s>115</s>|<s>file_mouselocklost.html</s>|<s>diogogmt</s>||||||<s>rhung</s>||<s><b>PEER REVIEWED, AND ADDED TO REPOSITORY</b></s>|||<s>Original file didn't need any changes. Passed all test criterias</s><s>Awaiting peer review to cross it off the list</s>|-|<s>119</s>|<s>file_fullscreen.html</s>|<s>northWind</s>|<s>X</s>|<s>X</s>|<s>X</s>|<s>X</s>|<s>X</s>|<s>rhung</s>||<s><b>FIXED, PEER REVIEWED, AND ADDED TO REPOSITORY</b></s>|X||<s>Lines</s> <s>49</s>, <s>66</s><s>, and 76 are too long. 76 in particular.</s> <s>Needs comment at the beginning to describe what the test does.</s>|-|<s>107</s>|<s>file_screenXYclientXYConst.html</s>|<s>diogogmt</s>|<s>X</s>|<s>X</s>||||<s>anachid</s>||<s><b>FIXED, PEER REVIEWED, AND ADDED TO REPOSITORY</b></s>|||<s>[https://github.com/diogogmt/mozilla-central/blob/mouselock-tests-constantXY/dom/tests/mochitest/mouselock/file_constantXY.html file_constantXY]</s><s>[https://github.com/diogogmt/mozilla-central/tree/mouselock-tests-constantXY branch]</s> <s>* Anachid (formatting review on file commit version '''1f5fa3e0a5f7730f33eb5d3f7c51105fac4cb93d'''):</s><s>** Line 8 exceeds 80 character limit by 7 characters.</s><s>** Remove line 99 [ console.log("start"); ]</s><s>** Line 100 and 103 could be re-done to document.getElementById("div") instead of having a div variable, thereby removing variable space and risk of a bug</s><s>** Extra line breaks (should be removed) on line 74,77,78</s>|-|108|file_defaultUnlock.html|diogogmt||||||||||||-|<s>103</s>|<s>file_DOMtree.html</s>|<s>diogogmt</s>||||||<s>rhung</s>||<s><b>FIXED, PEER REVIEWED, AND ADDED TO REPOSITORY</b></s>||||-|112|file_limitlessScroll.html|rhung|X||||rhung: Needs to be redone to take into account where the mouse currently is rather than sending it a totally new point.|||'''FIXING'''||||-|104|file_MouseEvents.html|rhung|X||X|||northWind||'''PEER REVIEWED, SEE COMMENTS'''|||* northWind:** hangs intermittently, most likely requires focus, see [[#Waiting For Focus|here]]** 39,41: not required, handled by harness** 51: remove line, setup being called at better time @343** 60: should use === for identity as opposed to ==** 60: should add document.mozFullScreen && to condition** 61, 69: needs failure callback with is(false, ...) and SimpleTest.finish()** 68: should add && document.mozFullScreenElement ===outer to condition** 68: whitespace at end of line** <s>78: looked at this briefly, are we sure that synthMs will cause the cursor to cross over the 'one' div?</s>** 172: Col 45: "the" should be "a Left"** 183-214: Why not testing for mousemove?|-|116|file_movementXY.html|diogogmt|||||| mjschranz||<b>REVIEWED, READY FOR PULL REQUEST</b>|||[https://github.com/diogogmt/mozilla-central/blob/mouselock-tests/dom/tests/mochitest/pointerlock/file_movementXY.html file_movementXY]* mjschranz** As far as I can tell, there doesn't appear to be any styling errors. The tests run just fine and based on what I understand about it and what is listed in the comments the tests appear to function as intended.|-|<s>120</s>|<s>file_syntheticMouseEvent.html</s>|<s>diogogmt</s>||||||<s>rhung</s>|||||<s>Merged with MouseEvents test</s>|-|105|file_targetOutOfFocus.html|diogogmt|||||| mjschranz||<b>REVIEWED, SEE COMMENTS</b>|||[https://github.com/diogogmt/mozilla-central/blob/mouselock-tests/dom/tests/mochitest/pointerlock/file_targetOutOfFocus.html file_targetOutOfFocus]* mjschranz** Line 41: the word focus is surrounded by single quotation marks, should be double ( "" ).** No other styling issues seen. Tests run and appear to be functioning as per the specs.|-|118|file_userPref.html||||X||X|||||||-|121|test_mouseLockable.html(rename to file_mozPointerLock.html)|||X|X|X|X|||||||-|122|file_nestedFullScreen.html|diogogmt||||X|| mjschranz||<b>REVIEWED, SEE COMMENTS</b>|||[https://github.com/diogogmt/mozilla-central/blob/mouselock-tests/dom/tests/mochitest/pointerlock/file_nestedFullScreen.html file_nestedFullScreen]* mjschranz** Line 48: isLocked method call is missing opening/closing brackets. The test still passes when FullScreenHarness is run however we obviously need to fix that and ensure it's still locked as intended.** No other styling issues seen. Fix previous error and double check all tests still pass when harness is run.|-|<s>123</s>|<s>file_unlockOnUnload.html</s>|<s>northWind</s>||||X||<s>rhung</s>||<s>'''FIXED, PEER REVIEWED, AND ADDED TO REPOSITORY'''</s>||||-|<s>120</s>|<s>file_differentDOM.html</s>|<s>diogogmt</s>||||||<s>rhung</s>||<s>'''FIXED, PEER REVIEWED, AND ADDED TO REPOSITORY'''</s>||||-|} <!--====Test Reviews[DEPRECATED]==== The following tests need review and/or fixes, and to then get updated in [https://github.com/rhung/mozilla-central/tree/mouselock-tests/dom/tests/mochitest/mouselock rhung's mouslock-tests] branch on github. Please add your name beside test(s) you are reviewing/fixing. Once a test is complete and updated in his branch, please cross it off below.{| border=1 style="border: 1px solid darkgray;"| Num| Name| Owner| Has Harness| Rev.| Peer Rev.| Pull| Contrib.|- | colspan=8 bgcolor="yellow" style="text-align:center;" | '''Ensure you have actually tested your test before sending a pull request'''|-|- | colspan=8 bgcolor="lightgreen" style="text-align:center;" | '''Fill Out the Contrib cell with "First Last <email>" or leave blank for none.'''|-|1|test_FullScreenHarness.html|Anachid|X|X|humph|X|Ching Wei Tseng (Steven) <steven_tseng15@hotmail.com>|-|2|mouselock_util.js|Anachid|X|X|humph|X|Ching Wei Tseng (Steven) <steven_tseng15@hotmail.com>|-|3|file_DOMtree.html|Anachid|X|X|humph|X| Ching Wei Tseng (Steven) <steven_tseng15@hotmail.com>|-|4|test_MouseEvents.html|rhung|X|X|jboelen|X|Raymond Hung <hung.raymond@gmail.com>|-|5|test_TargetOutOfFocus.html|johnno|||diogogmt|||-|6|test_UserAgentIsActive|johnno|||diogogmt|||-|7|test_constantxy.html|jbraffoul|X|X|Anachid|X|Ching Wei Tseng (Steven) <steven_tseng15@hotmail.com>, Jordan Raffoul <raffoul.jordan@gmail.com>|-|8|test_defaultUnlock.html|KeyR|X|X|nm486|X|Stanley Tsang <nm486@hotmail.com> Keyan Ren <rockyrky@hotmail.com>|-|9|test_doubleLockCallBack.html|jboelen||X|rhung||James Boelen <james@boelen.ca>|-|10|test_exitMouselockOnLoseFocus.html|drigato / johnno|||diogogmt|||-|11|test_isInstanceofMouselockable.html|abhatnagar|N/A|X|hchun|X|Hyungryul Chun <hello.ryul@gmail.com>, Anurag Bhatnagar <bhatnagar.anurag@gmail.com>|-|12|file_limitlessScroll.html|hchun|X|X|rhung, nm486|X|Hyungryul Chun <hello.ryul@gmail.com>, Stanley Tsang <nm486@hotmail.com>, Raymond Hung <hung.raymond@gmail.com>|-|13|test_mouseLockableHasRequiredMethods.html |abhatnagar|N/A|X|hchun|X|Hyungryul Chun <hello.ryul@gmail.com>, Anurag Bhatnagar <bhatnagar.anurag@gmail.com>|-|14|test_mousePos.html|||||||-|15|test_mouselocklost.html|nm486|X|X|hchun|X|Stanley Tsang <nm486@hotmail.com>, Hyungryul Chun (Steven) <hello.ryul@gmail.com>|-|16|file_movement.html|hchun|X|X|rhung, nm486|X|Hyungryul Chun <hello.ryul@gmail.com>, Stanley Tsang <nm486@hotmail.com>, Raymond Hung <hung.raymond@gmail.com>|-|17|test_navigatorPointer.html|abhatnagar||||||-|18|test_userPref.html|mjschranz|X|X|jboelen/rhung|X|Matthew Schranz <schranz.m@gmail.com>|-|19|test_fullscreen.html|abhatnagar1|x|x|||Abhishek Bhatnagar <abhatnagar1@learn.senecac.on.ca>|-|20|test_syntheticMouseEvent.html|Tentacle|x|x|Anachid|x|Qian Xu <qxu26@learn.senecac.on.ca>, Ching Wei Tseng (Steven) <steven_tseng15@hotmail.com>|} '''[DEPRECATED]'''# test_FullScreenHarness.html - Anachid# mouselock_util.js - Anachid# file_DOMtree.html - Anachid# <s>test_LockWhenOutOfFocus.html</s> Note: Removed# test_MouseEvents.html - rhung# <s>test_TargetOutOfFocus.html</s> - johnno# test_UserAgentIsActive.html - johnno# test_constantxy.html - jbraffoul# <s>test_defaultUnlock.html</s># <s>test_doesunlock.html</s> Note: Removed, redundant;# <s>test_doubleLockCallBack.html</s># <s>test_exitMouselockOnLoseFocus.html</s> drigato,johnno(merged)# <s>test_fullscreen.html</s># <s>test_isInstanceofMouselockable.html</s> - abhatnagar# <s>test_limitlessScroll.html</s> - hchun# <s>test_lockLostCallBack.html</s># <s>test_mouseLockableHasRequiredMethods.html</s> - abhatnagar# test_mousePos.html# <s>test_mouselocklost.html</s> - nm486# <s>test_movement.html</s> -hchun# <s>test_navigatorPointer.html</s> - abhatnagar# <s>test_syntheticMouseEvent.html</s># <s>test_userPref.html</s> - mjschranz--> ====Waiting For Focus====Many tests will intermittently hang if they don't wait for focus. To have a test wait for focus, ensure that the test incorporates the following template:<pre><!-- snip --><body onload="runTests();"> <!-- other tags that you may need go here --> <script> function runTests() { SimpleTest.waitForExplicitFinish(); SimpleTest.waitForFocus(function() { // Test code goes here }); } </script></body><!-- /snip --></pre> ====Using the Test Harness===='''Note: This harness assumes that your test runs properly as a stand-alone application and is not being used as one of the test amongst others.''' The problem with the test timing out is due to the fact that the Mochitest '''iframe does not allow fullscreen''' from the html code inside the frame. To solve this problem, it needs a small work around like using the harness. To make use of the harness, you will need to add the '''mouselock_util.js''' file to your test html file. I would recommend renaming your file to '''file_something.html''' (vs. test_something.html), as this denotes that this not a direct test under the mochitest. The same applies to the Makefile.in. '''Change the filename to their appropriate name in the Makefile.in''' Thus, add the following line, AFTER the EventUtil.js and SimpleTest.js script tag <pre><script type="application/javascript" src="mouselock_util.js"></script></pre> Another, thing that needs to be done is to add your file to the '''gTestFiles''' array inside the '''test_FullScreenHarness.html''' file After that, do a run of the whole mochitest directory on your machine before doing a pull request.(use this):<pre>TEST_PATH=dom/tests/mochitest/mouselock/ make -C $(replace with OBJDIR) mochitest-plain</pre> Finally, since the fullscreen needs the harness, the harness will provide the special powers. So please do '''NOT add any special powers''' in your child files. ==Demos, Docs, Other====
# Proper IDL documentation for navigator.pointer (see example in https://mxr.mozilla.org/mozilla-central/source/dom/interfaces/html/nsIDOMHTMLElement.idl#103), MouseLockable and its methods, MouseLockLost event, etc.
# Write a JavaScript library to somehow combine element.mozRequestFullScreen() and navigator.pointer.lock(). It would be good to hide the complexities of doing fullscreen then locking in a single API call. - nm486 <pre>var canvas = document.getElementById('canvas'); // Will put element into fullscreen then try to do mouse lockvar lock = acquireMouseLock({ element: canvas, onLock: function() { // OPTIONAL // fullscreen + mouse lock are now setup for element }, onUnlock: function() { // OPTIONAL // the user or browser took us out of fullscreen/mouse lock }, onError: function() { // OPTIONAL // there was an error getting fullscreen/mouse lock }, onMovemennt: function(e) { // OPTIONAL // movementX and movementY for mousemove are passed in var deltaX = e.movementX; var deltaY = e.movementY; }}); ...lock.islocked; // true or falselock.unlock();</pre>
# Mouse lock specification fix requests
# Convert Rescue Fox to use Mouse Lock, see https://github.com/mozilla/rescuefox
* Points of Interest: http://zenit.senecac.on.ca/wiki/index.php/Mouse_Lock_API_Points_of_Interest
* Mochitest: https://developer.mozilla.org/en/Mochitest
* Mochitest FAQ: http://zenit.senecac.on.ca/wiki/index.php/Mochitest_FAQ
* Mozilla Cross Reference: http://mxr.mozilla.org/mozilla-central/
* MouseLock Demos Link: http://humphd.github.com/mozilla-central/mouselock/index.html
* MouseLock Stage Build download: http://stage.mozilla.org/pub/mozilla.org/firefox/try-builds/david.humphrey@senecac.on.ca-f4e5849f1ae7/