[OPEN-ILS-DEV] Thoughts about the Staff Client Part 1 of 4 - TT2 and AngularJS

[Bill Erickson]

Hi Liam,

Thanks for taking the time with weigh in.  I can tell you put a lot of
thought into this and I appreciate it.  I'll address some of your points
individually below and in the other emails as I have time.

Before I dive in, I would like everyone to understand that a lot of thought
went into the design of the prototype.  The use of TT is no exception.  We
share similar concerns and the items you mention below, certainly for me,
were mulled extensively before deciding to use TT.

First a little history...

There are two styles of TT use in Evergreen.  There's the TPAC approach
where the server collects data and generates fully-realized HTML pages,
complete with data (search results, account info, etc.)  Then, there's the
staff web style, where TT generates an HTML page with no data and links to
Javascript (e.g. Dojo, Angular, etc.), which is then responsible for
collecting and rendering the content.

The latter approach has been in use with Evergreen since about 2007 for
generating base HTML pages for all of the JS driven staff client interfaces
(acq, conify, vandelay, serials, etc.).  It serves two important purposes:

1. server-side i18n generation.
2. server-side template inclusion and nested templates.

No JS framework, no matter how zazzy, can do all of these things and they
need doing, because they reduce network overhead.  Server-side i18n reduces
the number of external resources the client has to retrieve (e.g. string
bundles).  Server-side template inclusion and nesting reduces both network
overhead and code duplication, since it allows us to create templates once
and include them in the body of the page and it lets us create wrapper
pages so we'er not duplicating piles of boilerplate with each UI.

(Angular will let you share (unnested) templates, but only if you fetch
them as external resources.  There are times when this style of template
inclusion is a good thing.  For example, if the template is rarely
accessed, it makes little sense to make the client download it every time.
 I leverage this some in the prototype.  It's a great tool to have.  But,
if we want a way for clients to load shared templates within the body of
the page to reduce network overhead, then we need something that operates
on the server side, like TT).

What this approach does not do is pull data from the DB and build pages
from it.  That's a critical distinction and, I think, largely resolves many
of the issues raised below.

More inline below...

On Wed, Jan 22, 2014 at 7:18 PM, Liam Whalen
<liam.whalen at bc.libraries.coop>wrote:

> I am going to send four emails about my thoughts on the staff client.  I
> have not had time to investigate the prototype code very much yet, but I
> think some of the issues I am thinking about are things that should be
> considered before full development is begun regardless of the technical
> aspects of the code.
>
>
I would really appreciate it if you would review the code and, while you're
in there, think about how you would solve the problems TT solves for us
without using TT.  I really don't mean that to sound snarky,  I'm being
completely serious.  I'm not strongly tied to any in particular technology,
as I believe in the right tool for the job, but if we remove TT, then we
will have a new set of problems that AngularJS cannot solve and the
solution will be either to not solve them or use something... like TT.



> I am concerned that combining the use of TT2 with AngularJS adds
> complexity to the staff client design that will inhibit participation with
> staff client development, adds a layer of complexity to the MVC desgin, and
> could end up making the app incompatible with later versions of the
> AnuglarJS framework.
>

Using TT does add a layer of indirection.  Personally, I detest complexity.
 I would rather have fewer features than a system that cannot be
maintained.  In light of this, I still chose to use TT, because it solves
more problems that it creates.

What's more, the way we use TT in the staff interfaces is not full-blown
Template Toolkit.  We don't fetch data, we don't use loops or data
structures.  99% of the TT syntax is INCLUDE and [% l('translateable
string') %].  If those are too complicated, then Angular will blow a
developer's mind.

Finally, Evergreen UI developers that don't understand the basics (and just
the basics) of TT will be a rare bread, since we use it extensively (TPAC,
other staff UIs, Action/Trigger templates).



>
> By using TT2 with AngularJS, we are no longer using the AngularJS
> framework, we are now creating a hybrid system. In essence, we are rolling
> our own staff client by using multiple processes.  For developers outside
> of the Evergreen community who like using AngularJS, having to learn TT2
> could result in those developers losing interest in assisting with the
> staff client.
>

I see where you're coming from here and I share this concern, but in our
case, it's simply not true. We are using two frameworks within two distinct
problem domains.

Domain 1 is delivering HTML to the client.  Domain 2 is having the client
generate rich content from the base HTML.  TT takes templates as input and
produces HTML as output.  Angular takes HTML as input and produces rich
content in the browser as output.  There is no cross-over here.  The fact
that Angular receives HTML which was generated by some mysterious
server-side process is wholly irrelevant to Angular.  The HTML could have
come from a thousand monkeys behind a thousand typewriters (sorry, going
for humor), so long as the final product was an HTML page Angular could
drive.



>
> As well, the complexity of the Model View Control design is increased when
> TT2 processing defines the AngularJS controllers and directives that could
> be used on a page.  At this point, the developer needs to examine the Perl
> code to see what the values of these variables are at the time of
> processing.
>

No Perl programming is involved.  The developer is only be dealing with
templates, which are mostly plain HTML.


> Instead of having an MVC system that is self contained, where the models
> are easily identifiable within the files of the AngularJS code base, and
> the relations between those models and the view via controllers can be
> discerned by reading the AngularJS code, developers now have to maintain a
> secondary list of conditions defined outside of AngularJS that vary
> depending on some criteria such as the URL being used.
>

I think you are picturing an arrangement that is significantly more
complicated than what we are doing.

We are doing things like this:

<div ng-controller="foo"><span>Hello world</span></div>

v.s.

<div ng-controller="foo"><span>[% l('Hello world') %]</span></div>


>
> Also, for those unfamiliar with MVC, it may be tempting to make things
> work by using variables via TT2 that do not make sense in the context of
> the file that the person is editing.  In this case, the code still works,
> but the abstraction starts to fall apart and becomes a hinderance rather
> than an asset when thinking about the code.  I do not think the community
> has the time to police this sort of development, and best practices listed
> on a Wiki may not get followed.
>

Agreed, if we do things that don't make sense, problems will follow.  We
can solve that by being good developers that understand the system we are
developing for.  If the system is too complex, then yes, let's change the
system.

More to the point, though, TT will not be used to generate data.  There may
be some allure there, but to take that step would require new Perl modules
and new Apache configuration.  It's not something that happens on accident.
 We have intentionally designed the code so that TT variables, driven by
data from the database, simply don't exist.  (With one very important
exception we discussed in IRC -- the list of locales).


>
> Additionally, Google is talking about developing server side parsing of
> some AngularJS code.  While this will unlikely be mandatory for an
> AngularJS app in the near term, the benefits added by server side
> processing may be deemed important in new versions of the AngularJS
> framework.  In this case, a later version of the framework may require
> server side processing in order to use newer features.  Which would put the
> community in the position of either figuring out how to process something
> via TT2 then send it to server side AngularJS (and this may not be possible
> without an extreme effort), or we have to start limiting what we can use in
> the newer versions of the framework or are limited to using only older
> versions of the framework.
>
>
> I realize this is very much a what-if scenario, but I think the possible
> problems are important enough for it to be considered before large scale
> development combining TT2 and AngularJS commences.
>

> In the case of content being created via TT2, I am less concerned because
> that can be refactored fairly easily and without the fear of breaking the
> application. I am thinking of things like localization and custom messages.
>

Ironically, server-side JS-driven page generation is meant to solve some of
the same problems we have already solved by leveraging TT, namely providing
the ability to re-use templates without requiring the client to fetch them
as separate downloads.

In any event, I'm not worried about it.  If we decide to go this route,
then presumably this process would live on the server, maybe as an Apache
filter, or some other standalone process.  In any case, we can ask Apache
first for the TT-generated HTML and then run that through the
server-side-angular generator, whatever form it takes, and then pass the
result off to the client.


>
> Also, using TT2 as a preprocess before AngularJS adds the possibility that
> the template may inject Angular syntax into a piece of content by mistake.
> Errors like this would be very hard to track down and a solution (other
> than procedures to verify that content template values are not AngularJS
> code) is not generalizable for fixing something like this.  Again, this is
> an edge case, but it adds an element of unknown to all debugging that will
> have to be considered at some point once the source of an error eludes a
> developer for a long period of time.
>

Agreed, if we make mistakes, problems will follow.  Ditto above about
understanding the code we writing.


>
> In the end, by combining TT2 and AngularJS we are no longer using a
> framework.  At this point we are rolling our own solution via the
> combination of two separate systesm.  I am going to touch on Google Best
> practices in another email, but in general, the advice I hear from the
> AngularJS team communications is develop everything in AngularJS to the
> fullest extent possible.
>

I see this as a warning against mixing Angular with other JS frameworks, a
sentiment a heartily agree with.

Also, let's be clear that Angular is not magical.  It's just another JS
framework, one that I happen to like a lot.  The people that wrote it are
smart and they have a lot of wisdom to share, which we should soak up, but
they are not here, now, working on Evergreen and do not have to live with
these decisions.  Let's learn from them, but take our own paths where it
makes sense to.

Thanks again, Liam.  Let me know if I've said anything crazy.  I'm really
glad we can hash all of this out with the community before diving into the
project.

-b

-- 
Bill Erickson
| Senior Software Developer
| phone: 877-OPEN-ILS (673-6457)
| email: berick at esilibrary.com
| web: http://esilibrary.com
| Equinox Software, Inc. / The Open Source Experts
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20140123/b3adfcbf/attachment-0001.htm>