1
edit
Changes
no edit summary
== Project Name ==
''automated localization toolAutomatic Localization Fork Tool'' - This toolscript, given a locale and a set of rules, will create a build an l10n source tree of the same language but different region (e.g. en-IN from en-GB)* '''THE BUG''' - [https://bugzilla.mozilla.org/show_bug.cgi?id=403215 Bug 403215 – Python script that will help create from a translated locale a regional build]* <strike>'''THE BUG''' - [https://bugzilla.mozilla.org/show_bug.cgi?id=399014 Bug 399014 – we need an l10n-merge tool]</strike>
== How to Add to Tool & Pick It Up as a Project Description ==For those wondering how they can pick this project up and run with it, please read the following tips:<br />The code is broken into three main components* '''THE BUGcallback()''' - [https://bugzillais where you would add your new functionality and subsequently update the "instring" variable before closing the localization file.mozillaThis method is the most important part of the program and is essentially the core from which all localization logic is derived from.org/show_bugI added much in-code documentation to guide newcomers along the way.cgi?id=399014 Bug 399014 – we need an l10n-merge tool]* Learn python'''translate()''' is where you tell the program what files to look for and subsequently what action Parser.py (the class supplied by Axel) should take, such as use a DTD Parser or use a Properties parser and so on...* Understand '''process()''' contains the file iteration logic whereby the program scans every file in every directory. There is potential for optomization here and Dynamis (Tomoya Asai, a Mozilla employee from Japan) can definately help steer you in the scripts right direction as this method was inspired from reading the test logic in Dynamis' l10n tools-merge script.* Understand '''main()''' is where you would alter the program's main interface, adding UI code in here is a sure possibility.*'''The #l10n build systemIRC Channel on Moznet''' has a wealth of people who are knowledgable in this area and can provide much information if asked* Reproduce en-IN from en-GB'''Regular Expressions''' are you friend. Do what I did, go print out every copy of Regular Expression theory in Pyhon as you can, cause you will need it* Determine what our Python based system '''Mozilla MXR''' is your gateway to finding out where you will "do" in 0.1 releasebe looking to localize specific functionality or visual items
== Project Plan & Description ===== Project Description ===The l10n Auto Fork – formerly known as the Automated Localization Build Tool's – is a Python-based tool that, given a locale and set of rules, creates an l10n source tree similar to the original as far as the language, yet modified to be used in a specific region. A practical example would be to use an en-US source tree and run the l10n auto fork tool to produce an en-CA source tree. The tool would, among other things, modify the default RSS feeds, dictionary, as well as the search engines to the target locale. Essentially, the auto fork tool will generate a localization template - which would be in the form of an XML, DTD, or Properties document - that “localizers” can use to enter the localized version of each string, date, currency, or other form of data. When this template is provided to the tool along with a source tree (such as en-GB), the tool outputs the desired localized source tree.=== Project Plan ===There are seven main points to address on the road that takes this project from its current 0.3 all the way to an 1.0 release. Along the way I plan to keep in touch with Mic and Axel as well as the rest of the localization community through IRC, email, phone, and Bugzilla.The table below outlines what my goals for each bi-weekly release are right up to the final 1.0release.2 <br />{| border="1"|-| '''Release'''| '''Download Release Functionality & '''| '''Features ==/Points Addressed'''|-| 0.4|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.4.zip Download]* | - <strike>All regular expressions, and related functions removed and replaced in existing code. Optimize existing code (the current directory “walk” methods can be re-factored). Add new command line options. For example, an option to generate a localization template (won't be fully functional yet, completed at future release). Another option would be to have verbose mode on/off for the tool so localizers can see what files are being localized in real-time.</strike> '''INSTRUCTIONS(DONE)'''|-| 0.5|[http: //zenit.senecac.on.ca/wiki/imgs/L10nFork0.5.zip Download]| - <strike>Complete fixing outstanding bugs.</strike> '''(DONE)''' <strike> Help command (will display all command line arguments and switches that can be used with tool).</strike> '''(DONE IN 0.4)'''TBD. <strike>Validation code.</strike> '''(DONE)''' <strike>Generate a template as well as settle on a specific file format (dtd, properties, or XML).</strike> '''(DONE)'''|- | 0.6|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.6.zip Download]* | - <strike>Complete localization template functionality. Will provide “localizers” with a hassle free way of altering strings/dates/currency/default search engines/default RSS, to target locale.</strike>'''(DONE)'''|- | 0.7.2|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.7.2 release files.zip Download]| - <strike>Add feature that will allow to modify default search engines to target locale.</strike>'''(DONE)'''. <strike>Test on full source tree - worked, ScreenShots here </strike>'''(DONE)'''|-| 0.8.1|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.8.1.zip Download]| - <strike>Add feature that allows to change default dictionary</strike> '''(DONE)'''.|-| 0.9|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.9.zip Download]| - <strike>Seperate browser Strings & Labels localization template from browser Search Defaults localization template</strike>'''(DONE)''' <strike>Fixed a bug (if the first char in KEY was different from first char in VALUE, localization changes would not take place)</strike>'''(DONE)''' <strike>Improved code's readability</strike>'''(DONE)'''|-| 1.0|[http: //zenit.senecac.on.ca/wiki/imgs/L10nFork1.0.zip Download]| - <strike>Find way to potentially optomize localization process. Also, add functionality so localization file for search engines is auto generated. Added documentation on this wiki on how students or community can pick up this project and continue it.</strike>''TBD'(DONE)'''|-|}
== Tool'Project Contributor(s 0.1 Release Functionality & Features )==* '''INSTRUCTIONS''': [http://matrixThe following people contributed to our project in some beneficial manner.senecacWe appreciate all advice received from the Mozilla l10n community, as well as Seneca students, and professors.onIf you would like to be contributor to our project, just get in touch with us through email, or seneca's IRC channel.ca/~azambran/mozilla/instructions0.1.txt TXT file]* The '''0.1 release ZIP fileContribution Info''': [https://bugzillasub-section below outlines our current project needs.mozillaPotential contributors can pick one and we can talk about how to proceed.org/attachment.cgi?id=284789 The Zip file filed on the bug]
* Our project can benefit from anyone who is knowledgeable in Python regular expressionsVijey Balasundaram **Tested our tool and looked for bugs or any potential future pitfalls we might run into. Feel free to contact us if you would like to contribute to the project in this wayHelped with some documentation for first time tool users.
* We have a problem when matching a pattern inside of the DTDs, look below: re.sub(r'([Cc])olor', r'\1olour', instring)Mozilla l10n Community Changes every appearance of 'Color' or 'color' with 'Colour' and 'colour' (the Canadian wa) Therefore a line like: * <!ENTITY colorsDialog.title "Colors"> becomes like this: * <!ENTITY coloursDialogProvided strong leadership and support.title "Colours"> WE DONWe wouldn'T WANT colo'''u'''rsDialogt be able to get this project off the ground without them.title!!! A better regular expression could avoid this but we are not good yet with RegExps (check anlaysis on RegExprs on next section)
* We would like to walk into subfolders Seneca Professors **Put us in contact with the Mozilla l10n community and make changes in each subfolder; any good python example?helped us get this project off the ground through structure and guidance.
== Project Leader(s) == Contribution Info ====*[[User:Rueen|Rueen Fiez]] <strike>I would like to apply to a string multiple regular expressions substitutions. I have seen something like re.compile(rueenbla bla bla) . Could I collect a set of regular expressions in an array an iterate through it applying the substitutions to a string?? or something like that?</strike>*<strike>[http://rueenfiezzenit.wordpresssenecac.com Rueen's blog]*[[User:Vlam6|Vincent Lam]] (vlam6) [http:on.ca/wiki/vlam6index.wordpress.com Vince's blog]*[[php/User:Armenzg|Armen Zambrano]Tjduavis Timothy Joseph Duavis] (armenzghere to be of service!) [http:<//armenzg.blogspot.com Armenstrike> ('''Done'''s blog])
== Other Project Details ==* <strike>Our script for [http://docs.google.com/Doc?id=dnkkbhp_65p4whj5 now]; We will be also posting on the bug 399014</strike> - Get our code from the latest release* <strike>We are also awaiting for some code that dynamis has been working on in Japan</strike> - dynamys seems to be lost in combat - I have heard he might me moving to another location and that might be he has been difficult to contact him
* [http://docs.google.com/Doc?id=dnkkbhp_36gps298 Notes from Axel(pike)] about the project
* [http://docs.google.com/Doc?id=dnkkbhp_35gscqgh Team notes] - we collect notes related to the project
* [http://docs.google.com/Doc?id=dnkkbhp_32ds8tz7 Armen's MozDev process] - diary - You can read notes of what Armen has been trying
* The l10n tools are in [http://mxr.mozilla.org/mozilla/source/testing/tests/l10n/ mxr.m.o/mozilla/source/testing/l10n]
* How to check out the l10n tools using CVS: <pre>cvs -z3 -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/tools/l10n mozilla/testing/tests/l10n</pre>
* The file we have been using: [http://mxr.mozilla.org/mozilla/source/testing/tests/l10n/lib/Mozilla/Parser.py Parser.py] using the DTDParser class and the PropertiesParser class. These both are derived from the Parser class.
* [http://mxr.mozilla.org/mozilla/source/tools/l10n/l10n.py mozilla/tools/l10n/l10n.py] might give you an idea of what it takes to '''copy existing data over to a new location'''
* To get the l10n tools type: '''$> cvs -z3 -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/tools/l10n mozilla/testing/tests/l10n'''
* Some notes from trying to get en-GB ([http://armenzg.blogspot.com/2007/10/session-6-trying-to-build-en-gb.html read more]):
>> Customize -> Customise
==== DTD regular expression analysis ====
* An analysis of regular expression, specific to DTDs (from Parser.py)
self.key = re.compile('<!ENTITY\s+([\w\.]+)\s+(\"(?:[^\"]*\")|(?:\'[^\']*)\')\s*>', re.S)
"<!ENTITY colorsDialog.title "Colors">"
==== Related regular expressions theory ==== * (...) what is inside the parentheses are a group - the contents of a group can be retrieved after a match has been performed, and can be matched later in the string with the ''\number'' special sequence * (?...) - This is an '''extension notation''' - Extensions usually do not create a new group; (?P<name>...) is the only exception to this rule. Following are the currently supported extensions. * List of supported extensions: (?iLmsux), (?:...), (?P<name>...), (?P=name), (?#...), (?=...), (?!...), (?<=...), (?<!...), (?(id/name)yes-pattern|no-pattern)
* \number - Matches the contents of the group of the same number. Groups are numbered starting from 1.
== Bugs ==
* '''THE BUG''' - There are no bugs related specifically to the project, but the following were mentioned in our conversations[https://bugzilla.mozilla.org/show_bug.cgi?id=403215 Bug 403215 – Python script that will help create from a translated locale a regional build]* * <strike>'''THE BUG''' </strike> - [https://bugzilla.mozilla.org/show_bug.cgi?id=399014 Bug 399014 – we need an l10n-merge tool]* [https://bugzilla.mozilla.org/show_bug.cgi?id=398954 Bug 398954 – l10n build automation doesn't pick the right en-US source for the build]* [https://bugzilla.mozilla.org/show_bug.cgi?id=391680 Bug 391680 – Make the python compare-locales work with suite and calendar]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=345039 Bug 345039 en-CA English (Canada): Firefox] - please read comment 16
* [https://bugzilla.mozilla.org/show_bug.cgi?id=392945#c8 Bug 392945 (fx20-en-IN) – add en-IN locale]
== Project ContributorOld want to help ==* <strike> We have a problem when matching a pattern inside of the DTDs, look below:re.sub(r'([Cc])olor', r'\1olour', instring)Changes every appearance of 'Color' or 'color' with 'Colour' and 'colour' (the Canadian wa)Therefore a line like:<!ENTITY colorsDialog.title "Colors">becomes like this:<!ENTITY '''colours'''Dialog.title "Colours">WE DON'T WANT colo'''u'''rsDialog.title!!!A better regular expression could avoid this but we are not good yet with RegExps (scheck anlaysis on RegExprs on next section) </strike>'''SOLVED:''' re.sub(r'<font color='red'>(.*)</font>([Cc])olor', r'\1<font color='red'>\2</font>olour', instring) * <strike>We would like to walk into subfolders and make changes in each subfolder; any good python example? </strike> '''SOLVED:''' Already works on 0.2 release* <strike> Learn python </strike>* <strike> Understand the scripts from the test l10n tools </strike>* <strike> Understand the l10n build system </strike>* <strike> Reproduce en-IN from en-GB </strike>* <strike> Determine what our Python based system will "do" in 0.1 release </strike>