[Evergreen-general] Automating Antora documentation builds on GitLab

[Floyd, Angelia Lynn]

As I like this concept and the lowering of the bar for edits, my main concern is that we currently have people adding docs two different ways.

  1.  Straight to the working Git Repo
  2.  Via GitHub

Will the use of GitLab incorporate both of these methods for adding docs?

I think definitely a further discussion on this process and how to best use it at the next DIG meeting is a must.

-----------------------------
Lynn Floyd
MIS Supervisor
Indiana State Library
317-232-3290
lfloyd1 at library.in.gov<mailto:lfloyd1 at library.in.gov>

From: Evergreen-general <evergreen-general-bounces at list.evergreen-ils.org> On Behalf Of Dan Scott via Evergreen-general
Sent: Thursday, December 30, 2021 11:45 AM
To: evergreen-general at list.evergreen-ils.org
Subject: Re: [Evergreen-general] Automating Antora documentation builds on GitLab

**** This is an EXTERNAL email. Exercise caution. DO NOT open attachments or click links from unknown senders or unexpected email. ****
________________________________
I took things a little further today.

The GitLab docs build now includes the Evergreen CSS, header, and footer. The UI bundle is currently sourced from a pinned job from an automated build of the "evergreen" branch of the Antora default UI (https://gitlab.com/denials/antora-ui-default/-/tree/evergreen [1]) I set up.

The UI bundle is currently in my GitLab namespace for testing purposes but it would be easy to move the repository to GitLab's evergreen-library-system group. I suggest mirroring the Antora/antora-ui-default main branch (GitLab makes mirroring trivial) and keeping our customizations in an "evergreen" branch to avoid merge hassles, etc.

If we were willing to rely on GitLab's automated builds, we could remove docs/generate_docs.pl<https://protect2.fireeye.com/v1/url?k=b33ef7c2-eca5cedd-b33abec2-8630ffab37ab-13f03a9685753a65&q=1&e=7b0675b3-8f20-4e04-83dc-82cedabdf6e5&u=http%3A%2F%2Fgenerate_docs.pl%2F> and replace docs/site.yml. We wouldn't need to maintain a special server to build and serve the Evergreen documentation. We could even have GitLab serve up the docs.evergreen-ils.org<https://protect2.fireeye.com/v1/url?k=5ae76b66-057c5279-5ae32266-8630ffab37ab-2e16b86f003fd1ac&q=1&e=7b0675b3-8f20-4e04-83dc-82cedabdf6e5&u=http%3A%2F%2Fdocs.evergreen-ils.org%2F> domain, given that Evergreen 3.6 marks the first Antora-based documentation and is currently in "only security fix mode". Or we could continue to serve up docs.evergreen-ils.org<https://protect2.fireeye.com/v1/url?k=4b8ac5b6-1411fca9-4b8e8cb6-8630ffab37ab-d8bb1bad123ab804&q=1&e=7b0675b3-8f20-4e04-83dc-82cedabdf6e5&u=http%3A%2F%2Fdocs.evergreen-ils.org%2F> on our own server along with the legacy docs by automating the downloading & unzipping of the artifacts bundle (all the HTML/CSS/JS/media) GitLab creates with each build.

Depending on how far we wanted to go with GitLab integration, we could also unhide the "Edit this page" link on every page of the Evergreen documentation that leads to the corresponding file in the GitLab repository and the web IDE. That might be a lower bar for basic documentation contributions (e.g. fixing typos or adding a clarification) than our current footer's suggestion to email the DIG list (which, based on the list archives, doesn't seem to be attracting many contributions). Edits made via the Web IDE create a merge request (& potentially a branch) per https://docs.gitlab.com/ee/user/project/web_ide/#commit-changes so contributors wouldn't have to learn git mechanics to edit existing files.

On Wed, 29 Dec 2021 at 14:30, Dan Scott <dan at coffeecode.net<mailto:dan at coffeecode.net>> wrote:
I was exploring GitLab's CI/CD (continuous integration / delivery) functionality last night/this morning and decided to try building our Antora-based documentation this way with the output hosted on GitLab Pages.

It works quite nicely. I've posted a rough implementation branch at https://gitlab.com/evergreen-library-system/evergreen-library-system/-/commits/gitlab-antora-build

As you can see, it's just a couple of YAML files; .gitlab-cy.yml defines the "pages" step, and .gitlab-antora.yml defines a custom Antora playbook for the Docker environment. The CI/CD piece means that a new build would be triggered by every commit.

The output is visible at https://evergreen-library-system.gitlab.io/evergreen-library-system/docs/latest/index.html<https://protect2.fireeye.com/v1/url?k=caca5588-95516c97-cace1c88-8630ffab37ab-97c482cf18007337&q=1&e=7b0675b3-8f20-4e04-83dc-82cedabdf6e5&u=https%3A%2F%2Fevergreen-library-system.gitlab.io%2Fevergreen-library-system%2Fdocs%2Flatest%2Findex.html>

It's nice to see the current list of build errors, such as missing references or media files, at a glance: https://gitlab.com/evergreen-library-system/evergreen-library-system/-/jobs/1927489146

If we decided to merge this, we would want to modify the configuration so that it only watches a given branch (our main branch, most likely), rather than triggering a new build for every commit  on every branch. But most of the doc build could also be reused as an integration test to be run against every branch / merge request. If a test fails, then the merge request gets flagged, the errors get reported as a comment right on the branch/merge request (instead of having to dig into the job details), and the branch could be blocked from merging until the problem is resolved.

I'm using the Node.js npx command to avoid having to globally install the npm modules. npx is a nice way to avoid conflicts between requirements for different module versions at the global level and to avoid having to run commands as root. When you prefix a command with "npx", such as "npx antora", it searches for the command within the local project's node_modules path instead of globally. Could be useful elsewhere in Evergreen.

Notes:
* I updated the build process to use the just-released Antora 3 and the corresponding Antora lunr-extension.
* I have not applied the Evergreen customizations of CSS, the header menu, or the contact DIG message in the footer. Antora 3 has moved to "extensions" which are lighter weight than our current approach of cloning and tweaking the Antora default UI repository.
* In addition to the basic "does Antora build the docs without errors" integration test, we could add a style checker like Vale (https://gitlab.com/jdkato/vale) as another test for the documentation. Vale can flag spelling errors (with a custom dictionary of course) and grammatical issues like passive voice, etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.evergreen-ils.org/pipermail/evergreen-general/attachments/20211230/50d65582/attachment-0001.html>