Portable DXR/meeting 110508
Date : Nov 5, 2008
Topic : Wide ranging discussion about progress, milestones, and technical requirements
People: James Boston (jboston), Samer Ziadeh (samer), Jerry Pau (jPau), David Humphrey (humph)
13:39 < jboston> humph: I just I'll start by asking if you have seen the updates to the wiki or my latest blog post
and have any insults. I mean comments.
13:39 <@humph> I havne't seen your blog post
13:39 <@humph> is it on planet now?
13:39 < jboston> It should be.
13:39 <@humph> oh I see it now
13:39 * humph reads
13:40 < jboston> MLM seems pleased with what we've done, although we've done it behind schedule.
13:40 < samer> it felt good though a positive reaction for once
13:41 < jPau> And she wants us to keep the momentum..
13:41 < samer> let's increase the momentum
13:41 < jboston> I think we need more kinetic energy.
13:42 <@humph> so one thing I notice
13:42 < jboston> ?
13:42 < jboston> It's all wrong?
13:42 <@humph> I don't think updates will happen via the web content in the app
13:42 <@humph> the app itself will update
13:42 <@humph> i.e., you won't trigger it via some extjs gui call
13:42 < samer> oh it happens by itself?
13:42 < jboston> Welll, I think the app would notify when updates are available.
13:42 <@humph> "The last part is the the presentation of the documentation, the DXR proper. David has a something
now that uses XHTML, Ext-JS for the front-end. We could port that to the extension, but there would
need to be modifications for sure. For instance, David's DXR doesn't give the user the option of
updating. If we are combining this with Prism and XULRunner, the only UI elements the user will see
will be in the displayed page. So we would need to include...
13:42 <@humph> that is wrong, I think
13:43 <@humph> you want the entire app to update, which will pull your new content, and then when it starts the web
server in the ext, it will get index.html from within the ext dir
13:43 < samer> another issue
13:43 < jboston> But if it's prism, Will there be any chrome for an update button?
13:43 < jPau> The app will online and check for new updates
13:43 < samer> if they have slow internet, and the download is a few hundred MB then they need to be connected for
that while
13:43 < samer> a few hours or so
13:43 <@humph> how do you update your firefox now?
13:43 <@humph> do you press a button?
13:44 <@humph> (you can, but do you?)
13:44 < jPau> They force you?
13:44 <@humph> it polls and sees an update is there
13:44 < samer> yeah, I select the update now or update later button
13:44 < jboston> Well, no. There's a window opens.
13:44 <@humph> the ext has an update URL it checks
13:44 < samer> but the download happens in the bg
13:44 <@humph> it checks at various points and then when an update is there, asks you to install it
13:44 < samer> so we need a sort of pause/resume download thing
13:44 <@humph> no
13:45 < jboston> See, I'm thinking of updating the app and updating the extension as two different things.
13:45 <@humph> you need to figure out how to use the built-in ext/xulrunner autoupdate stuff
13:45 <@humph> I see what you mean
13:45 <@humph> is it different?
13:45 <@humph> maybe there are 2 extensions: 1 for web server + 1 for web content, extjs, sqlite db, etc.
13:45 < samer> I thought the ext was the app :/
13:45 < jboston> It could be. Espcially if the extension is general purpose.
13:45 <@humph> I think the first ext is very general purpose
13:46 <@humph> esp. if you can hand it another ext as a package of content
13:46 <@humph> but then, question:
13:46 < jPau> Would you want to adapt an autoupdate? What if the user is getting off his 56k in 2 minutes?
13:46 <@humph> jPau: solved issue. mozilla has solved all this.
13:46 <@humph> just use it
13:46 < jPau> oh
13:46 <@humph> so let's say you do 2 ext
13:47 <@humph> the general one is a web server living in prism that is not bound to any content per se
13:47 <@humph> so you install multiple content extensions, pdxr being one
13:47 <@humph> how do you choose to run that vs. another one?
13:48 < samer> each ext is responsible for it's own actions
13:48 <@humph> there are really 2 problems here, and I'm not sure you should solve them both
13:48 < jboston> I was thinking about that. Is there a way to intercept a url and then resolve it to the localhost.
ie. the user loads dxr.com but it goes to local content
13:49 <@humph> I say do a tight binding between ext-web and ext-dxr-content, make them one
13:49 <@humph> jboston: an ext could do that, sure
13:49 <@humph> you can generalize this code later
13:49 < jboston> Great. See then, there could be an online/offline mode type thing.
13:49 <@humph> that's already built-into moz too
13:50 <@humph> they have something like gears in ff3
13:50 <@humph> for offline mode with a web app
13:50 <@humph> I mean, another way to do this is to pre-load the offline cache
13:50 <@humph> but
13:50 <@humph> you can't pre-load dynamic content
13:50 < jboston> Ah. Are we re-inventing google gears?
13:50 <@humph> I won't let youl, no
13:50 <@humph> this isn't a pure offline problem
13:50 <@humph> since you need to generate content (e.g., queries)
13:50 <@humph> so you need web+perl
13:51 < jboston> Oh, question about cgi!
13:51 <@humph> I think, to make your life easy you do this:
13:51 <@humph> second
13:51 < jboston> I never gave any consideration to the licencing of distributing perl interpreters.
13:52 <@humph> make an extension. this extension has a binary component, which is a web server + perl interpreter.
Also, there is a content part, which holds all your web pages, sqlite db, source code, your web app,
etc.
13:52 < jboston> ok
13:52 <@humph> so you install this ext into prism, and get the autoupdate code for an ext working
13:52 <@humph> then, when you fix the web server code, it pushes that. AND, when you update the source code files
or indexes, it pushes that too
13:53 <@humph> this reminds me that you'll have an issue
13:53 <@humph> the glimpse indexes (which mxr uses to do full text search) are absolute paths
13:53 <@humph> we'll need to solve that somehow in order to get it anchored in the extension install dir
13:54 < jboston> ok
13:54 <@humph> can you make a note of that?
13:54 <@humph> it will come back to haunt us
13:54 < jboston> Notes.
13:55 <@humph> so, someone needs to learn how to do xulrunner autoupdating
13:55 <@humph> Dave Townsend (Mossop) did a talk on this at the dev day
13:55 <@humph> and some day I'll get teh video back :(
13:56 <@humph> when your web server gets started, you'll point it at index.html within the extension install dir
13:56 <@humph> you follow?
13:56 <@humph> jboston: re: perl license, yes, that's key to solve
13:56 < jboston> I've looked again at shttpd. It comes with a header file for compiling against other applications.
Maybe it's still the best choice. Very lightweight. The interpreters aren't built in. I think we
can swap out what it there for something more robust.
13:56 <@humph> I suspect you'll have choices
13:56 <@humph> I think it's one to consider, yes
13:57 <@humph> if you do an IDL in front of it, nsILocalWebServer, you can do an abstraction so you can do different
backends
13:57 <@humph> much like nsProcess is to nsIProcess
13:57 < jboston> The perl interpreter bacially reads stdout from the server and then sends a webpage back to the
server's stdin?
13:57 <@humph> yeah
13:58 < jboston> For some reason I thought something more exotic was happening.
13:58 <@humph> it's just files, yeah
13:58 <@humph> your nsIProcess piping may be helpful here
13:58 < jboston> So we need a lightweight but robust interpreter that we can compile across three platforms and
legally distribute.
13:59 <@humph> focus on windows
13:59 <@humph> since all of what we're doing is free on linux/mac (e.g., both ship with apache + perl)
13:59 <@humph> windows is the pain point
13:59 <@humph> however, if it works on all 3, great
13:59 <@humph> just don't let that stop your choice
14:00 < jboston> Okay. I think that the interpreter doesn't need to be a binary xpcom component. It's something that
is started by the server per request if I understand it correctly.
14:00 <@humph> right
14:00 < jboston> Super.
14:00 <@humph> you could just ship something that lives in the ext dir
14:01 <@humph> could be a simple perl.exe
14:01 <@humph> where "simple" is not so simple, since you need libs
14:01 <@humph> in the worst case, you could install activestate perl
14:01 <@humph> and have your app ship with an installer
14:01 < jboston> That's cheating.
14:02 <@humph> that's 0.1
14:02 < samer> lol
14:02 <@humph> working code trumps lofty design
14:03 < jboston> Where would you like to see us by the end of this term in terms of deliverables since there is no
implementation per se.
14:03 <@humph> you mean this term
14:03 <@humph> ?
14:03 < jboston> yes.
14:03 < jboston> I guess great documentation?
14:04 <@humph> I'd like for you guys to be ready to code this beast
14:04 <@humph> so you need to know your web server
14:04 <@humph> perl bit
14:04 <@humph> need to understand the update mechanism
14:04 <@humph> maybe make a small throw-away xulrunner/prism app that autoupdates
14:04 < jPau> hmm. do you have a list of suggestion that i can add onto our wbs?
14:04 < jboston> We were thinking that we could try setting up this dehydra thing to play around with, see what it
spits out.
14:04 < jPau> and do i have to extend to the winter semester?
14:05 <@humph> yes, let's get that done too
14:05 < jboston> jPau: we can ask MLM about the ms project stuff i think
14:05 < jPau> i'm trying to follow and understand the stuff thats mentioned on here
14:05 <@humph> what I'd also like is this:
14:05 < jPau> yeah
14:05 < samer> agh yes the ms project
14:05 <@humph> a series of bi-weekly release targets that you'll meet over the second term
14:06 <@humph> 0.1 == ..., 0.2 == ...
14:06 <@humph> I don't care about MS project personally
14:06 <@humph> she might, though
14:06 <@humph> but don't do it for me
14:06 < samer> we could find an alternative for her
14:07 < samer> she just wants ms project because that's the way it's been
14:07 <@humph> the wiki with milestone releases would be what I want
14:07 <@humph> but I'm open
14:07 < samer> but we havea different project so we can show her timelines in different ways
14:07 <@humph> note: I won't be installing MS project
14:07 < samer> yeah we'll do the wiki with milestones
14:07 <@humph> jPau: I think you need to get some mozilla experience under your belt
14:07 <@humph> you're going to get a real shock when you start coding this otherwise
14:08 < jboston> humph: Perhaps we could meet and you could show how you install dehydra. It might help us
understand what we are pushing as updates.
14:08 <@humph> sure, that's a good use of time
14:08 < samer> yes
14:08 <@humph> let's replicate my setup on one of the cdot boxes
14:09 <@humph> and you can play with that install
14:09 < samer> cool
14:13 <@humph> jPau: you with us?
14:13 < jPau> Yeah. I do need to get some mozilla experience.. Do you know where I can start?
14:13 <@humph> I have some ideas
14:14 < samer> lol
14:14 < samer> I think he's talking about his class
14:14 <@humph> I think you 3 need to also draw some clean lines as to who will do what
14:14 <@humph> that will help us get skills paired with work
14:15 < jPau> Well. If we are coding in PERL, i can do that.. But when it comes to interfacing with Mozilla, I think
I will be a bit lost..
14:15 <@humph> that's not going to cut it
14:15 <@humph> so I'm going to leave this with you 3 for our next meeting
14:15 < samer> ok
14:15 < jboston> ok
14:15 < jPau> ok
14:15 <@humph> I want to understand the division of labour, and how we get people on the track to winning
14:16 < samer> hmm
14:17 < samer> don't think we really discussed that
14:17 <@humph> that's very important here
14:17 <@humph> I don't want "We will all work on X"
14:17 <@humph> that's crap
14:17 <@humph> 3 people == 3 different parallel tasks
14:18 < jboston> Concurrency is a difficult problem to solve.
14:18 <@humph> setting out your release milestones will help with this
14:18 <@humph> only if you try to post back to the main thread
14:18 < samer> so what are we looking for M1
14:18 * humph is pretty sure that's what you're supposed to figure out now in this course
14:22 <@humph> so are we done?
14:22 < samer> I think so
14:22 < jboston> Ok. Thanks for the time and advice.
14:22 < samer> Thanks for the meet
14:22 <@humph> ok