''automated localization toolAutomatic Localization Fork Tool'' - This tool should allow that script, given a locale and a set of changes (unknown how) gives a build rules, will create an l10n source tree of the same language but different region (e.g. en-IN from en-GB)* '''THE BUG''' - [https://bugzilla.mozilla.org/show_bug.cgi?id=403215 Bug 403215 – Python script that will help create from a translated locale a regional build]* <strike>'''THE BUG''' - [https://bugzilla.mozilla.org/show_bug.cgi?id=399014 Bug 399014 – we need an l10n-merge tool]</strike>

== How to Add to Tool & Pick It Up as a Project Description ==For those wondering how they can pick this project up and run with it, please read the following tips:<br />The code is broken into three main components* '''THE BUGcallback()''' - [https://bugzillais where you would add your new functionality and subsequently update the "instring" variable before closing the localization file.mozillaThis method is the most important part of the program and is essentially the core from which all localization logic is derived from.org/show_bugI added much in-code documentation to guide newcomers along the way.cgi?id=399014 Bug 399014 – we need an l10n-merge tool]* Learn python'''translate()''' is where you tell the program what files to look for and subsequently what action Parser.py (the class supplied by Axel) should take, such as use a DTD Parser or use a Properties parser and so on...* Understand '''process()''' contains the file iteration logic whereby the program scans every file in every directory. There is potential for optomization here and Dynamis (Tomoya Asai, a Mozilla employee from Japan) can definately help steer you in the scripts right direction as this method was inspired from reading the test logic in Dynamis' l10n tools-merge script.* Understand '''main()''' is where you would alter the program's main interface, adding UI code in here is a sure possibility.*'''The #l10n build systemIRC Channel on Moznet''' has a wealth of people who are knowledgable in this area and can provide much information if asked* Reproduce en-IN from en-GB'''Regular Expressions''' are you friend. Do what I did, go print out every copy of Regular Expression theory in Pyhon as you can, cause you will need it* Determine what our Python based system '''Mozilla MXR''' is your gateway to finding out where you will "do" in 0.1 releasebe looking to localize specific functionality or visual items

== Project Plan & Description ===== Project Description ===The l10n Auto Fork – formerly known as the Automated Localization Build Tool– is a Python-based tool that, given a locale and set of rules, creates an l10n source tree similar to the original as far as the language, yet modified to be used in a specific region. A practical example would be to use an en-US source tree and run the l10n auto fork tool to produce an en-CA source tree. The tool would, among other things, modify the default RSS feeds, dictionary, as well as the search engines to the target locale. Essentially, the auto fork tool will generate a localization template - which would be in the form of an XML, DTD, or Properties document - that “localizers” can use to enter the localized version of each string, date, currency, or other form of data. When this template is provided to the tool along with a source tree (such as en-GB), the tool outputs the desired localized source tree.=== Project Plan ===There are seven main points to address on the road that takes this project from its current 0.3 all the way to an 1.0 release. Along the way I plan to keep in touch with Mic and Axel as well as the rest of the localization community through IRC, email, phone, and Bugzilla.The table below outlines what my goals for each bi-weekly release are right up to the final 1.0 release.<br />{| border="1"|-| '''Release's ''| '''Download Release'''| '''Features/Points Addressed'''|-| 0.4|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.4.zip Download]| - <strike>All regular expressions, and related functions removed and replaced in existing code. Optimize existing code (the current directory “walk” methods can be re-factored). Add new command line options. For example, an option to generate a localization template (won't be fully functional yet, completed at future release). Another option would be to have verbose mode on/off for the tool so localizers can see what files are being localized in real-time.</strike> '''(DONE)'''|-| 0.5|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.5.zip Download]| - <strike>Complete fixing outstanding bugs.</strike> '''(DONE)''' <strike> Help command (will display all command line arguments and switches that can be used with tool).</strike> '''(DONE IN 0.4)'''. <strike>Validation code.</strike> '''(DONE)''' <strike>Generate a template as well as settle on a specific file format (dtd, properties, or XML).</strike> '''(DONE)'''|- | 0.6|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.6.zip Download]| - <strike>Complete localization template functionality. Will provide “localizers” with a hassle free way of altering strings/dates/currency/default search engines/default RSS, to target locale.</strike>'''(DONE)'''|- | 0.7.2|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.7.2.zip Download]| - <strike>Add feature that will allow to modify default search engines to target locale.</strike>'''(DONE)'''. <strike>Test on full source tree - worked, ScreenShots here </strike>'''(DONE)'''|-| 0.8.1 Release Functionality |[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.8.1.zip Download]| - <strike>Add feature that allows to change default dictionary</strike> '''(DONE)'''.|-| 0.9|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork0.9.zip Download]| - <strike>Seperate browser Strings & Features ==Labels localization template from browser Search Defaults localization template</strike>'''(DONE)''' <strike>Fixed a bug (if the first char in KEY was different from first char in VALUE, localization changes would not take place)</strike>'''(DONE)''' <strike>Improved code's readability</strike>'''(DONE)'''|-| 1.0|[http://zenit.senecac.on.ca/wiki/imgs/L10nFork1.0.zip Download]| - <strike>Find way to potentially optomize localization process. Also, add functionality so localization file for search engines is auto generated. Added documentation on this wiki on how students or community can pick up this project and continue it.</strike>'''(DONE)'''|-|}

== Project Leader(s) ==*[[User:Rueen|Rueen Fiez]] (rueen) [http://rueenfiez.wordpress.com Rueen's blog] - (Point of Contact)*[[User:Vlam6|Vincent Lam]] (vlam6) [http://vlam6.wordpress.com Vince's blog]*[[User:Armenzg|Armen Zambrano]] (armenzg) [http://armenzg.blogspot.com/search/label/auto%20l10n Armen's blog about the project] == Releases ===== Release notes ===* Should be able 1.0 Release'''(DONE)'''** <strike>Find way to accept a potentially optomize localization process. Also, add functionality so localization filefor search engines is auto generated</strike>* Should be able to accept <strike> 0.9 Release** Seperate browser Strings & Labels localization template from browser Search Defaults localization template** Fixed a Firefox build bug (eg; en-GB or en-USif the first char in KEY was different from first char in VALUE, localization changes would not take place)* Stores file* Improved code's data locally (need readability</strike>*<strike> 0.8 Release** Add feature that allows to change default dictionary </strike>*<strike>0.7 Release** Add feature that will allow to modify default search engines to target locale.** Test on full source tree - worked, ScreenShots here. </strike>*<strike>0.6 Release** Complete localization template functionality. Will provide “localizers” with a hassle free way of altering strings/dates/currency/default search engines/default RSS, to use IO if target locale.</strike>*<strike>0.5 Release** Complete fixing outstanding bugs** Validation code** If time permits, get the template generator functional.txt file ''(orThat way its one less thing todo for 0.7 release) XML Parser if it's an ' </strike>*<strike>0.4 release** Remove regex code and supporting methods** Optimize existing methods such as the directory walk method.xml file(Small thing but necessary nonetheless)** Add new command line options such as an option to generate localization template and an option to turn on tool's verbose mode</strike>*<strike>0. So this would be taking 3 release</strike>** <strike>Use of command line arguments rather than receiving user's input</strike>** <strike>Fix the keychanging problem</value pairs and storing them strike>** <strike>Add more words to change in a Dictionary an l10n tree. But they are hard-coded :(</strike> ** <strike>Stop using .bak file (Dictionary is and make the equivalent changes permanent)</strike>** <strike>Generate a second l10n tree out of original with the changes</strike>*<strike>0.2 release</strike>** <strike>Given any directory as a Map in Javastarting point, should walk through all sub-directories and files and make changes based on translation rules</strike>** <strike>Add ability to update Properties files</strike>** <strike>Fix key changing problem ('''In progress/May go to 0.3 release''')</strike>** <strike>Add more "rules", for now we are testing by changing ''color'' --> ''colour'' ('''May go to 0.3 release''')</strike>** <strike>Allow user to enter localization folder (eg; en-GB) as input</strike> *<strike>0.1 release</strike>** <strike>Should be able to accept a localization</strike>** <strike>Should be able to accept an l10n tree(eg; en-GB or something along those lines.en-US)</strike>* * <strike>Read through every DTD and Properties file in the current directory with the "Parser.py" file</strike>* * <strike>Changes the word "color" to "colour" in every DTD file and have it saved</strike> == Project news ==There are some common news from the collaborators that should be written here rather than splitting it between the collaborators:* Apr. 18, 2008 - Release 1.0 is now available, localizing process is now optomized to be completed faster. New functionality also added, ability to automatically generate a localization template for the default search engines with predefined keys* Apr. 6, 2008 - Release 0.9 - Big change to the way localizer enters their changes. They must now enter String/Label localization changes in the "locale_template.properties" file and enter Search Default changes in the "locale_template_searches.properties" file. This seperation increases readability and avoids confusion when entering many localization values. Also fixed a critical bug found related to the first char of KEY being different than the first char of VALUE in properties files. * Mar. 23, 2008 - Release 0.8 - tool will now alter values in Firefox's default dictionary (this means even the spellchecker in Firefox will localize accordingly) * Mar. 9, 2008 - Updated 0.7 to 0.7.2 after running tests on a full source tree. Screenshots available here.* Feb. 29, 2008 - Release 0.7 is done. I added the ability to change the locale of the default search engines - located in '''/browser/locales/en-US/searchplugins/'''.* Feb. 20, 2008 - Release 0.6 is completed. The tool is now able to make changes dynamically based on the Key/Value pairs located in the Localization Template .properties file. This file must be located in the same directory as the tool itself for the tool to be able to output a new localized source tree.* Jan. 29, 2008 - Updated [https://bugzilla.mozilla.org/show_bug.cgi?id=403215 Bug 403215] with information regarding latest release as well as plans for future releases. Updated release features for 0.5 and 0.7.* Jan. 20, 2008 - Release 0.4 is completed. Program has now been turned into a ''real'' command line tool. Added several command line options. Can view them all by running '''"l10nFork.py -help"'''. Fixed a MAJOR bug left over from 0.3.2, the localized changes were not taking place, bug was found in the callback() function. Removed/Commented-out all Regex, left one since need it for testing purposes right now. By 0.7 Release, 100% of regex in callback() function will be removed.* Jan. 10, 2008 - Project's roadmap from '''0.3''' release to a '''1.0''' release posted. New releases will be available on a bi-weekly basis. Outline of new features for each new release available in the [http://zenit.senecac.on.ca/wiki/index.php/Automated_localization_build_tool#Project_Plan_.26_Description Project Plan & Description section]. Many general Wiki page updates. The tool is now called the Automated Localization Fork tool, or l10n Auto Fork for short.* Nov. 11, 2007 - Updated the contributions page.* Nov. 10, 2007 - l10nMerge tool updated from '''0.2.2''' to '''0.2.3''' and available for download under the 0.2 Release Functionality & Features section. Added new feature (allow user to directly input localization [eg; en-GB] ), added lots of code documentation and debug sections.* Nov. 9, 2007 - Created new bug (403215) that will guide the discussion about our tool* Oct. 31, 2007 - It seems that a lot of things are going to be talked around l10n tools in general ([http://groups.google.com/group/mozilla.dev.l10n/browse_thread/thread/61f5742491a4d3fc " L10n tools talking" in Google Groups]) and specifically in the bug ([https://bugzilla.mozilla.org/show_bug.cgi?id=399014 Bug 399014]) ''"related"'' (between brackets) to our project. We will have to read dynamis' (he has finally appeared, he is called Asai Tomoya) code and see what the new direction of our script should take. <strike>'''WE ARE A LITTLE CONFUSED BUT AFTER THE WEEKEND SOME LIGHT MIGHT DROP INTO THE PROJECT'''</strike> ('''Done''')* Oct. 30, 2007 - Axel has replied on the bug and it seems that '''our project shouldn't be called l10n merge''', the bug seems to be for another reason. ''Some doubts arise concerning what our tool should really be doing''.* Oct. 25, 2007 - Meeting with Michal; She explained us the different tools that out there created by the community and by next week will gives us more information on which direction to follow* Oct. 23, 2007 - Added 0.2 Release Functionality & Features section. Team meeting held (6 hours) - our tool is now able to, given a localization directory (or any directory for that matter), walk through all sub-directories and files and make changes based on our translation rules. We've also determined a few other things to do before our 0.2 release and even brainstormed 0.3 ideas (incorporating l10n's setup.py script into our tool like Axel suggested).* Oct. 12, 2007 - Updated 0.1 Release Functionality & Features section. A lot of the 0.1 code has been done.* Oct. 07, 2007 - Added an 0.1 Release Functionality & Features section to the wiki so we have a clear description of what our project's 0.1 release should be able to do.* Oct. 05, 2007 - Python will be our language of choice for this project which is a great opportunity to thoroughly learn it since it will be our first time using it. Determined some main tasks ahead of us before 0.1 release (tasks mentioned in [http://zenit.senecac.on.ca/wiki/index.php/Automated_localization_build_tool#Project_Description Product Description]).* Sep. 24, 2007 - We are going to have a call conference with Michal from Toronto office == Project Contributor(s)==The following people contributed to our project in some beneficial manner. We appreciate all advice received from the Mozilla l10n community, as well as Seneca students, and professors. If you would like to be contributor to our project, just get in touch with us through email, or seneca's IRC channel. The '''Contribution Info''' sub-section below outlines our current project needs. Potential contributors can pick one and we can talk about how to proceed. === Potential Ways of Contributing ===<strike>Python regular expressions help</strike><strike><u>Bug:</u> When parsing files that use key/value pairs - '''DTD or Properties''' - our tool not only changes the value but the key as well. We don't want the key to be changed.</strike>#Testing our script when needed. (Usually right after a release)

== Want * Timothy Joseph Duavis **Reviewed our Python script(s) and provided some ways to help??? ==improve several methods. In particular, the process() and callback() methods.

* Our project can benefit from anyone who is knowledgeable in Python regular expressionsVijey Balasundaram **Tested our tool and looked for bugs or any potential future pitfalls we might run into. Feel free to contact us if you would like to contribute to the project in this wayHelped with some documentation for first time tool users.

# We have a problem when matching a pattern inside of the DTDs, look below: re.sub(r'([Cc])olor', r'\1olour', instring) Changes every appearance of 'Color' or 'color' with 'Colour' and 'colour' (the Canadian wa)* Mozilla l10n Community Therefore a line like: * <!ENTITY colorsDialog.title "Colors"> becomes like this: * <!ENTITY coloursDialogProvided strong leadership and support.title "Colours"> WE DONWe wouldn'T WANT colo'''u'''rsDialogt be able to get this project off the ground without them.title!!! A better regular expression could avoid this but we are not good yet with RegExps (check anlaysis on RegExprs on next section)

# We would like to walk into subfolders * Seneca Professors **Put us in contact with the Mozilla l10n community and make changes in each subfolder; any good python example?helped us get this project off the ground through structure and guidance.

== Project Leader(s) == Contribution Info ====*[[User:Rueen|Rueen Fiez]] <strike>I would like to apply to a string multiple regular expressions substitutions. I have seen something like re.compile(rueenbla bla bla) . Could I collect a set of regular expressions in an array an iterate through it applying the substitutions to a string?? or something like that?</strike>*<strike>[http://rueenfiezzenit.wordpresssenecac.com Rueen's blog]*[[User:Vlam6|Vincent Lam]] (vlam6) [http:on.ca/wiki/vlam6index.wordpress.com Vince's blog]*[[php/User:Armenzg|Armen Zambrano]Tjduavis Timothy Joseph Duavis] (armenzghere to be of service!) [http:<//armenzg.blogspot.com Armenstrike> ('''Done'''s blog])

== Other Project Details ==* <strike>Our script for [http://docs.google.com/Doc?id=dnkkbhp_65p4whj5 now]; We will be also posting on the bug 399014</strike> - Get our code from the latest release* <strike>We are also awaiting for some code that dynamis has been working on in Japan</strike> - dynamys seems to be lost in combat - I have heard he might me moving to another location and that might be he has been difficult to contact him

==== DTD regular expression analysis ====

==== Related regular expressions theory ==== * (...) what is inside the parentheses are a group - the contents of a group can be retrieved after a match has been performed, and can be matched later in the string with the ''\number'' special sequence * (?...) - This is an '''extension notation''' - Extensions usually do not create a new group; (?P<name>...) is the only exception to this rule. Following are the currently supported extensions. * List of supported extensions: (?iLmsux), (?:...), (?P<name>...), (?P=name), (?#...), (?=...), (?!...), (?<=...), (?<!...), (?(id/name)yes-pattern|no-pattern)

* '''THE BUG''' - There are no bugs related specifically to the project, but the following were mentioned in our conversations[https://bugzilla.mozilla.org/show_bug.cgi?id=403215 Bug 403215 – Python script that will help create from a translated locale a regional build]* * <strike>'''THE BUG''' </strike> - [https://bugzilla.mozilla.org/show_bug.cgi?id=399014 Bug 399014 – we need an l10n-merge tool]* [https://bugzilla.mozilla.org/show_bug.cgi?id=398954 Bug 398954 – l10n build automation doesn't pick the right en-US source for the build]* [https://bugzilla.mozilla.org/show_bug.cgi?id=391680 Bug 391680 – Make the python compare-locales work with suite and calendar]

== Project ContributorOld want to help ==* <strike> We have a problem when matching a pattern inside of the DTDs, look below:re.sub(r'([Cc])olor', r'\1olour', instring)Changes every appearance of 'Color' or 'color' with 'Colour' and 'colour' (the Canadian wa)Therefore a line like:<!ENTITY colorsDialog.title "Colors">becomes like this:<!ENTITY '''colours'''Dialog.title "Colours">WE DON'T WANT colo'''u'''rsDialog.title!!!A better regular expression could avoid this but we are not good yet with RegExps (scheck anlaysis on RegExprs on next section) </strike>'''SOLVED:''' re.sub(r'<font color='red'>(.*)</font>([Cc])olor', r'\1<font color='red'>\2</font>olour', instring) * <strike>We would like to walk into subfolders and make changes in each subfolder; any good python example? </strike> '''SOLVED:''' Already works on 0.2 release* <strike> Learn python </strike>* <strike> Understand the scripts from the test l10n tools </strike>* <strike> Understand the l10n build system </strike>* <strike> Reproduce en-IN from en-GB </strike>* <strike> Determine what our Python based system will "do" in 0.1 release </strike>