[OPEN-ILS-DEV] Thoughts about the Staff Client Part 1 of 4 - TT2 and AngularJS

[Liam Whalen]

On Jan 23, 2014, at 9:19 AM, Bill Erickson <berick at esilibrary.com> wrote:

> Thanks for taking the time with weigh in.  I can tell you put a lot of thought into this and I appreciate it.  I'll address some of your points individually below and in the other emails as I have time.
> 

My pleasure Bill!  I hope we can arrive at the best possible solution for the new staff client through this process.

> Before I dive in, I would like everyone to understand that a lot of thought went into the design of the prototype.  The use of TT is no exception.  We share similar concerns and the items you mention below, certainly for me, were mulled extensively before deciding to use TT.
> 
> First a little history...
> ...

Thank you for that information.

> More inline below...
> 
> On Wed, Jan 22, 2014 at 7:18 PM, Liam Whalen <liam.whalen at bc.libraries.coop> wrote:
> I am going to send four emails about my thoughts on the staff client.  I have not had time to investigate the prototype code very much yet, but I think some of the issues I am thinking about are things that should be considered before full development is begun regardless of the technical aspects of the code.
> 
> 
> I would really appreciate it if you would review the code and, while you're in there, think about how you would solve the problems TT solves for us without using TT.  I really don't mean that to sound snarky,  I'm being completely serious.  I'm not strongly tied to any in particular technology, as I believe in the right tool for the job, but if we remove TT, then we will have a new set of problems that AngularJS cannot solve and the solution will be either to not solve them or use something... like TT. 
> 

I started taking a closer look last night.  I will increase my efforts and give you a complete code review.  What kind of timeline are you dealing with?

I have also started thinking about how to provide a template processing via client side interaction.  One possible solution would use HTML5's Web Storage feature to store modified templates.  Then the client side can retrieve from Web Storage and if a template does not exist there, then it can retrieve the default from the server.  I have been thinking about this problem since the dev meeting, and it is one of the reasons I have not responded sooner.  I know I need to find a solution to the template problem before we can consider alternatives to mixing TT2 and AngularJS.

>  
> I am concerned that combining the use of TT2 with AngularJS adds complexity to the staff client design that will inhibit participation with staff client development, adds a layer of complexity to the MVC desgin, and could end up making the app incompatible with later versions of the AnuglarJS framework.
> 
> Using TT does add a layer of indirection.  Personally, I detest complexity.  I would rather have fewer features than a system that cannot be maintained.  In light of this, I still chose to use TT, because it solves more problems that it creates.
> 

I would say it solves more problems from the context of our current code base, but we have the chance to rebuild and redesign.  Assuming something can be done about templates and i18l support, I think using a pure AngularJS design will simplify the code base that developers have to work with.  A lot of AngularJS's benefits come from the use of conventions.  A standard directory structure, standard naming conventions, and standard design patterns are some examples of this.  By moving away from this we loose these benefits, which is fine from our stand point because we work with this code all the time, but if I was a pure AngualrJS developer, I would be very frustrated while trying to learn the differences.   The use of conventions is designed to allow developers to come up to speed on a project very quickly.  Without those conventions, some of AngularJS's benefits are lost.

> What's more, the way we use TT in the staff interfaces is not full-blown Template Toolkit.  We don't fetch data, we don't use loops or data structures.  99% of the TT syntax is INCLUDE and [% l('translateable string') %].  If those are too complicated, then Angular will blow a developer's mind.

I have seen a loop last night.

src/templates/staff/circ/checkin/t_checking_table.tt2 contains the following code.

<div ng-init="
checkins.setColumns([
[%- FOR col IN COLUMNS %]
{label:'[% col.label %]',name:'[% col.name %]'[% IF col.display %],display:true[% END %]}[% IF !loop.last; ','; END -%]
[% END %]
])">
</div>

This is using TT2 to populate the parameters for an AngularJS function.  This level of abstraction is confusing to me.

As well, TT2 variables are used to define  the values for ng-app.

[%- IF ctx.page_app %] ng-app="[% ctx.page_app %]"[% END -%]

This means, conceptually, the program is broken up into a number of separate AngularJS applications.

But practically, because everything is tied to the same abstracted functions these separate applications are tightly coupled to each other, which means modifying something like web/js/ui/defaults/staff/services/list.js will modify how every application is used.  And this is the point of something like list.js.  But, once you get a new View that cannot accommodate the current functionality in list.js then you need to expand the logic and branches in list.js, which results in an overly complex module.  My example of list.js may not be the best in this case because I have not yet reviewed the code.  But, my point is by coupling everything together it increases complexity and decrease our flexibility to modify particular aspects of the staff client without modifying others unless we want to make overly complex code to handle the different cases.

From what I have read of AngularJS, we should have a single ng-app and tie the various staff client functionality together via separate AngularJS modules.

Things like the use of server defined ng-app values diverge drastically from what I have seen in Google's examples.  I am pushing conformity with Google's practices because it gives us the best chance of staying compatible with later versions of AngularJS. I realize the case of ng-app being defined on the server is not an issue, at least while we are dealing purely with client side AngularJS, but AngularJS is changing rapidly, and I would not depend on that always being the case.

> 
> Finally, Evergreen UI developers that don't understand the basics (and just the basics) of TT will be a rare bread, since we use it extensively (TPAC, other staff UIs, Action/Trigger templates).
> 

I am more concerned with expanding the number of developers we have than with our current set of developers being unfamiliar with TT2.

>  
> 
> By using TT2 with AngularJS, we are no longer using the AngularJS framework, we are now creating a hybrid system. In essence, we are rolling our own staff client by using multiple processes.  For developers outside of the Evergreen community who like using AngularJS, having to learn TT2 could result in those developers losing interest in assisting with the staff client.
> 
> I see where you're coming from here and I share this concern, but in our case, it's simply not true. We are using two frameworks within two distinct problem domains.  
> 
> Domain 1 is delivering HTML to the client.  Domain 2 is having the client generate rich content from the base HTML.  TT takes templates as input and produces HTML as output.  Angular takes HTML as input and produces rich content in the browser as output.  There is no cross-over here.  The fact that Angular receives HTML which was generated by some mysterious server-side process is wholly irrelevant to Angular.  The HTML could have come from a thousand monkeys behind a thousand typewriters (sorry, going for humor), so long as the final product was an HTML page Angular could drive.
> 

Using AngularJS's javascript to run the staff client code is not the biggest benefit from adopting AngularJS.  AngularJS should provide us with a way to rapidly develop application features.  It is the code base that benefits from AngularJS' use.  In many respects, the need for the javascript library to parse the AngularJS syntax is a side effect of wanting to use AngularJS in our code base.  By mixing TT2 and AngularJS we loose things like being able to debug completely within a web browser.  The final client side code can be debugged, but an error may result from the server side interaction, which requires developers to maintain two levels of abstraction when working with the code.

As well, if I find an error in the Angular app.  I then have to got back and work with the combination of TT2 and AngularJS instead of being able to dive right into the Angular code where the error was detected.

>  
> 
> As well, the complexity of the Model View Control design is increased when TT2 processing defines the AngularJS controllers and directives that could be used on a page.  At this point, the developer needs to examine the Perl code to see what the values of these variables are at the time of processing.
> 
> No Perl programming is involved.  The developer is only be dealing with templates, which are mostly plain HTML.  

This was an error.  I was confusing the staff client with the TPAC, but my point remains the same.  The developer has to examine the TT2 code to understand what part of the AngularJS app they are working with.

> 
> 
> Instead of having an MVC system that is self contained, where the models are easily identifiable within the files of the AngularJS code base, and the relations between those models and the view via controllers can be discerned by reading the AngularJS code, developers now have to maintain a secondary list of conditions defined outside of AngularJS that vary depending on some criteria such as the URL being used.
> 
> I think you are picturing an arrangement that is significantly more complicated than what we are doing.
> 
> We are doing things like this:
> 
> <div ng-controller="foo"><span>Hello world</span></div>
> 
> v.s.
> 
> <div ng-controller="foo"><span>[% l('Hello world') %]</span></div>
>  

I am not too concerned with using TT2 for content.  But, as I mentioned above it is being used for more than that already.

> 
> Also, for those unfamiliar with MVC, it may be tempting to make things work by using variables via TT2 that do not make sense in the context of the file that the person is editing.  In this case, the code still works, but the abstraction starts to fall apart and becomes a hinderance rather than an asset when thinking about the code.  I do not think the community has the time to police this sort of development, and best practices listed on a Wiki may not get followed.
> 
> Agreed, if we do things that don't make sense, problems will follow.  We can solve that by being good developers that understand the system we are developing for.  If the system is too complex, then yes, let's change the system.  
> 
> More to the point, though, TT will not be used to generate data.  There may be some allure there, but to take that step would require new Perl modules and new Apache configuration.  It's not something that happens on accident.  We have intentionally designed the code so that TT variables, driven by data from the database, simply don't exist.  (With one very important exception we discussed in IRC -- the list of locales).  
>  

I guess I come from a more dictatorial style of development.  It is great if we can rely on all contributors being good developers, but I know I make poor choices for many reasons.  Sometimes time constraints do not allow me to think about how I should organize my code in the best possible manner. Other times, I am simply not smart enough or experienced enough to do things in the correct manner.

By combining TT2 and AngularJS we are giving developers more possible ways to produce bad code.

> 
> Additionally, Google is talking about developing server side parsing of some AngularJS code.  While this will unlikely be mandatory for an AngularJS app in the near term, the benefits added by server side processing may be deemed important in new versions of the AngularJS framework.  In this case, a later version of the framework may require server side processing in order to use newer features.  Which would put the community in the position of either figuring out how to process something via TT2 then send it to server side AngularJS (and this may not be possible without an extreme effort), or we have to start limiting what we can use in the newer versions of the framework or are limited to using only older versions of the framework.
> 
> I realize this is very much a what-if scenario, but I think the possible problems are important enough for it to be considered before large scale development combining TT2 and AngularJS commences.
> 
> In the case of content being created via TT2, I am less concerned because that can be refactored fairly easily and without the fear of breaking the application. I am thinking of things like localization and custom messages.
> 
> Ironically, server-side JS-driven page generation is meant to solve some of the same problems we have already solved by leveraging TT, namely providing the ability to re-use templates without requiring the client to fetch them as separate downloads. 
> 
> In any event, I'm not worried about it.  If we decide to go this route, then presumably this process would live on the server, maybe as an Apache filter, or some other standalone process.  In any case, we can ask Apache first for the TT-generated HTML and then run that through the server-side-angular generator, whatever form it takes, and then pass the result off to the client.  

I am not ready to make those presumptions.  We could end up in the same position we are currently in with our inability to use the newest versions of XUL.

>  
> 
> Also, using TT2 as a preprocess before AngularJS adds the possibility that the template may inject Angular syntax into a piece of content by mistake. Errors like this would be very hard to track down and a solution (other than procedures to verify that content template values are not AngularJS code) is not generalizable for fixing something like this.  Again, this is an edge case, but it adds an element of unknown to all debugging that will have to be considered at some point once the source of an error eludes a developer for a long period of time.
> 
> Agreed, if we make mistakes, problems will follow.  Ditto above about understanding the code we writing.
>  

My point is we are making it hard to understand the code we are writing by combining both AngularJS and TT2.  I like to keep as little information in my memory as possible.  If I have to remember how the AngularJS files are generated on the server in order to understand what I am debugging on the client side, it creates more possible points of failure as I program.

> 
> In the end, by combining TT2 and AngularJS we are no longer using a framework.  At this point we are rolling our own solution via the combination of two separate systesm.  I am going to touch on Google Best practices in another email, but in general, the advice I hear from the AngularJS team communications is develop everything in AngularJS to the fullest extent possible.
> 
> I see this as a warning against mixing Angular with other JS frameworks, a sentiment a heartily agree with.
> 
> Also, let's be clear that Angular is not magical.  It's just another JS framework, one that I happen to like a lot.  The people that wrote it are smart and they have a lot of wisdom to share, which we should soak up, but they are not here, now, working on Evergreen and do not have to live with these decisions.  Let's learn from them, but take our own paths where it makes sense to.  
> 

I disagree about taking our own path.  I think we are best served by using what Google's suggestions as close to 100% of the time as possible.

> Thanks again, Liam.  Let me know if I've said anything crazy.  I'm really glad we can hash all of this out with the community before diving into the project.
> 

You are welcome Bill.  Thank you for taking all the time you have on creating the prototype.  You have said nothing crazy.  I think a lot of this comes down to how much time are we, as a community, are willing to invest in redesigning the staff client.

Liam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20140123/5f863a33/attachment-0001.htm>