[OPEN-ILS-DEV] Thoughts about the Staff Client Part 1 of 4 - TT2 and AngularJS

[Bill Erickson]

On Thu, Jan 23, 2014 at 1:49 PM, Liam Whalen
<liam.whalen at bc.libraries.coop>wrote:

> On Jan 23, 2014, at 9:19 AM, Bill Erickson <berick at esilibrary.com> wrote:
>

[snip]

I started taking a closer look last night.  I will increase my efforts and
> give you a complete code review.  What kind of timeline are you dealing
> with?
>
> I have also started thinking about how to provide a template processing
> via client side interaction.  One possible solution would use HTML5's Web
> Storage feature to store modified templates.  Then the client side can
> retrieve from Web Storage and if a template does not exist there, then it
> can retrieve the default from the server.  I have been thinking about this
> problem since the dev meeting, and it is one of the reasons I have not
> responded sooner.  I know I need to find a solution to the template problem
> before we can consider alternatives to mixing TT2 and AngularJS.
>

Ah, yes, locally modified templates.  This is something TT excels at.
 (Can't believe I forgot to mention that).  TT is really good for
supporting (layers of) locally modified template overlays.   The
infrastructure is already in place and any Evergreen admin that has tweaked
TPAC templates will understand the concepts for modifying staff templates
-- and they all live in the same place on the server.


>
>
>> I am concerned that combining the use of TT2 with AngularJS adds
>> complexity to the staff client design that will inhibit participation with
>> staff client development, adds a layer of complexity to the MVC desgin, and
>> could end up making the app incompatible with later versions of the
>> AnuglarJS framework.
>>
>
> Using TT does add a layer of indirection.  Personally, I detest
> complexity.  I would rather have fewer features than a system that cannot
> be maintained.  In light of this, I still chose to use TT, because it
> solves more problems that it creates.
>
>
> I would say it solves more problems from the context of our current code
> base, but we have the chance to rebuild and redesign.  Assuming something
> can be done about templates and i18l support, I think using a pure
> AngularJS design will simplify the code base that developers have to work
> with.  A lot of AngularJS's benefits come from the use of conventions.  A
> standard directory structure, standard naming conventions, and standard
> design patterns are some examples of this.  By moving away from this we
> loose these benefits, which is fine from our stand point because we work
> with this code all the time, but if I was a pure AngualrJS developer, I
> would be very frustrated while trying to learn the differences.   The use
> of conventions is designed to allow developers to come up to speed on a
> project very quickly.  Without those conventions, some of AngularJS's
> benefits are lost.
>

Just to clarify, I'm not suggesting we leverage TT because it's there, I
think we should use it because it's a good idea.

Standard naming conventions and Angular design patterns are all in play
here.  The only difference is how the files are generated.  For the
directory structure, every coder with a blog claims to have designed the
best way to structure their Angular apps.  Is there a One True Way now?
 How many Angular apps are out there that didn't get that message in time?
 Are they doomed to failure?



>
> What's more, the way we use TT in the staff interfaces is not full-blown
> Template Toolkit.  We don't fetch data, we don't use loops or data
> structures.  99% of the TT syntax is INCLUDE and [% l('translateable
> string') %].  If those are too complicated, then Angular will blow a
> developer's mind.
>
>
> I have seen a loop last night.
>
> src/templates/staff/circ/checkin/t_checking_table.tt2 contains the
> following code.
>
> *<*div* ng-init=*"
> checkins.setColumns([
> [%- FOR *col* IN *COLUMNS* %]
> {label:'[% *col*.*label* %]',name:'[% *col*.*name* %]'[% IF *col*.
> *display* %],display:true[% END %]}[% IF !*loop*.*last*; ','; END -%]
> [% END %]
> ])"*>*
> *</*div*>*
>
>
Ah, yes, you got me there. I felt liberated to experiment pretty heavily in
the prototype as I was adding extra (unplanned) features.  This is from my
column-picker experiment.  It's ugly and needs to be burned with fire.




> This is using TT2 to populate the parameters for an AngularJS function.
>  This level of abstraction is confusing to me.
>
> As well, TT2 variables are used to define  the values for ng-app.
>
> [%- IF ctx.page_app %] *ng-app*="[% ctx.page_app %]"[% END -%]
>
>
This, to me, is a perfectly reasonable example of avoiding repetition.
 When a new app is started, set the app value, load the wrapper template,
and now I don't have to copy/paste 20 lines of boilerplate from another
html file.  You're going to have a difficult time convincing me this is a
bad thing.



> This means, conceptually, the program is broken up into a number of
> separate AngularJS applications.
>

Most definitely.


>
> But practically, because everything is tied to the same abstracted
> functions these separate applications are tightly coupled to each other,
> which means modifying something like
> web/js/ui/defaults/staff/services/list.js will modify how every application
> is used.
>

On the contrary, they are not tightly coupled.  There is a one-way
dependency, like any modular structure.  Any app that uses lists.js relies
on it and must change when/if list.js changes.  (After it settles in, it
should stop changing).  But the services themselves have no knowledge of
the apps.

FWIW, better examples of services to review would be the net.js, idl.js,
pcrud.js and others.  list.js is part of my nascent column-picker
experimentation.



> And this is the point of something like list.js.  But, once you get a new
> View that cannot accommodate the current functionality in list.js then you
> need to expand the logic and branches in list.js, which results in an
> overly complex module.  My example of list.js may not be the best in this
> case because I have not yet reviewed the code.  But, my point is by
> coupling everything together it increases complexity and decrease our
> flexibility to modify particular aspects of the staff client without
> modifying others unless we want to make overly complex code to handle the
> different cases.
>
> From what I have read of AngularJS, we should have a single ng-app and tie
> the various staff client functionality together via separate AngularJS
> modules.
>

 Yes, that is true up to a point.  After much deliberation and
experimentation, I opted to break from that particular mold.  Cramming
every UI from the staff client into a single Angular app would create a
memory gobbling monster of an application.  Even if you offload most of the
page-specific controller code (which would require some planning), making
the client download the full list of routes for every staff UI just to get
to the login page is unwise and completely unnecessary.  Then, once you
start using the application, as each page loaded, controllers are
instantiated, which adds to memory consumption and the memory would not
clear until you manually reloaded the page or closed the tab.

My solution was to collect logically grouped application features into
apps.  The use of HTML pushstate routing for building links between
different pages means navigating between apps is seamless.  When a user
navigates to another app, the memory from the previous app is released.  So
far, it's worked like a charm.  If there are specific concerns, I'd love to
hear them.


> Things like the use of server defined ng-app values diverge drastically
> from what I have seen in Google's examples.
>

Well, sure, they don't use TT, so the HTML is going to look different.  The
end product is the same thing you see in Google examples, though.

This (in Chrome) should look about the same:  view-source:
https://bill-dev2.esilibrary.com/eg/staff/circ/patron/search

We have an app, some controllers, some Angular template variables, etc.


> I am pushing conformity with Google's practices because it gives us the
> best chance of staying compatible with later versions of AngularJS.
>

Me too!  Which is why the HTML we produce via TT should conform to Angular
best practices.

I realize the case of ng-app being defined on the server is not an issue,
> at least while we are dealing purely with client side AngularJS, but
> AngularJS is changing rapidly, and I would not depend on that always being
> the case.
>

> Finally, Evergreen UI developers that don't understand the basics (and
> just the basics) of TT will be a rare bread, since we use it extensively
> (TPAC, other staff UIs, Action/Trigger templates).
>
>
> I am more concerned with expanding the number of developers we have than
> with our current set of developers being unfamiliar with TT2.
>

These developers will have to navigate a sea of new technology with
Evergreen and OpenSRF.  JSON-query, PCRUD, the IDL, how the tables of the
database work together, managing transactions, and a whole host of APIs
that return a wide array of constructs and data types.   If they've made it
that far, TT will be a welcome bit of simplicity.

There is no scenario where a developer joins the community and only
understands Angular and I think it's unwise to set that expectation.



>
>
>
>>
>> By using TT2 with AngularJS, we are no longer using the AngularJS
>> framework, we are now creating a hybrid system. In essence, we are rolling
>> our own staff client by using multiple processes.  For developers outside
>> of the Evergreen community who like using AngularJS, having to learn TT2
>> could result in those developers losing interest in assisting with the
>> staff client.
>>
>
> I see where you're coming from here and I share this concern, but in our
> case, it's simply not true. We are using two frameworks within two distinct
> problem domains.
>
> Domain 1 is delivering HTML to the client.  Domain 2 is having the client
> generate rich content from the base HTML.  TT takes templates as input and
> produces HTML as output.  Angular takes HTML as input and produces rich
> content in the browser as output.  There is no cross-over here.  The fact
> that Angular receives HTML which was generated by some mysterious
> server-side process is wholly irrelevant to Angular.  The HTML could have
> come from a thousand monkeys behind a thousand typewriters (sorry, going
> for humor), so long as the final product was an HTML page Angular could
> drive.
>
>
> Using AngularJS's javascript to run the staff client code is not the
> biggest benefit from adopting AngularJS.  AngularJS should provide us with
> a way to rapidly develop application features.  It is the code base that
> benefits from AngularJS' use.  In many respects, the need for the
> javascript library to parse the AngularJS syntax is a side effect of
> wanting to use AngularJS in our code base.  By mixing TT2 and AngularJS we
> loose things like being able to debug completely within a web browser.  The
> final client side code can be debugged, but an error may result from the
> server side interaction, which requires developers to maintain two levels
> of abstraction when working with the code.
>

> As well, if I find an error in the Angular app.  I then have to got back
> and work with the combination of TT2 and AngularJS instead of being able to
> dive right into the Angular code where the error was detected.
>

That's not true.  The error occurs in the client, with a line number,
referring to an HTML file, sitting in the browser, that has no TT syntax
within in.

Yes, if your TT syntax is bad, it will produce errors on the server.  This
file will never get to the browser and Angular / JS will never be involved.
 There is no situation where you will be confused about whether the problem
is with Angular or TT.

Now, if you mis-type the HTML in the template, then that has nothing to do
with TT.

[snip]


>> Additionally, Google is talking about developing server side parsing of
>> some AngularJS code.  While this will unlikely be mandatory for an
>> AngularJS app in the near term, the benefits added by server side
>> processing may be deemed important in new versions of the AngularJS
>> framework.  In this case, a later version of the framework may require
>> server side processing in order to use newer features.  Which would put the
>> community in the position of either figuring out how to process something
>> via TT2 then send it to server side AngularJS (and this may not be possible
>> without an extreme effort), or we have to start limiting what we can use in
>> the newer versions of the framework or are limited to using only older
>> versions of the framework.
>>
>> I realize this is very much a what-if scenario, but I think the possible
>> problems are important enough for it to be considered before large scale
>> development combining TT2 and AngularJS commences.
>>
>
>> In the case of content being created via TT2, I am less concerned because
>> that can be refactored fairly easily and without the fear of breaking the
>> application. I am thinking of things like localization and custom messages.
>>
>
> Ironically, server-side JS-driven page generation is meant to solve some
> of the same problems we have already solved by leveraging TT, namely
> providing the ability to re-use templates without requiring the client to
> fetch them as separate downloads.
>
> In any event, I'm not worried about it.  If we decide to go this route,
> then presumably this process would live on the server, maybe as an Apache
> filter, or some other standalone process.  In any case, we can ask Apache
> first for the TT-generated HTML and then run that through the
> server-side-angular generator, whatever form it takes, and then pass the
> result off to the client.
>
>
> I am not ready to make those presumptions.  We could end up in the same
> position we are currently in with our inability to use the newest versions
> of XUL.
>


Let me get this straight, if we don't hand-code every character of every
HTML file and put every file in some specific location, we risk the same
fate as, for example, XULRunner deprecating remote XUL?  If this is true,
then we should not use Angular.  No one should.

[snip]


>> In the end, by combining TT2 and AngularJS we are no longer using a
>> framework.  At this point we are rolling our own solution via the
>> combination of two separate systesm.  I am going to touch on Google Best
>> practices in another email, but in general, the advice I hear from the
>> AngularJS team communications is develop everything in AngularJS to the
>> fullest extent possible.
>>
>
> I see this as a warning against mixing Angular with other JS frameworks, a
> sentiment a heartily agree with.
>
> Also, let's be clear that Angular is not magical.  It's just another JS
> framework, one that I happen to like a lot.  The people that wrote it are
> smart and they have a lot of wisdom to share, which we should soak up, but
> they are not here, now, working on Evergreen and do not have to live with
> these decisions.  Let's learn from them, but take our own paths where it
> makes sense to.
>
>
> I disagree about taking our own path.  I think we are best served by using
> what Google's suggestions as close to 100% of the time as possible.
>

I don't discard their recommendations lightly.  I tried many times to
follow the recommended patterns, and in most ways I do.  I started out not
using TT, but at some point, the benefits were too great.  In each case
where I do leave the path, there is some precipitating factor, like the
knowledge that doing XYZ is going to slow everything down or require that
we write twice as much code.

-b

-- 
Bill Erickson
| Senior Software Developer
| phone: 877-OPEN-ILS (673-6457)
| email: berick at esilibrary.com
 | web: http://esilibrary.com
| Equinox Software, Inc. / The Open Source Experts
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20140124/b49ed1fa/attachment-0001.htm>