Hugo website build and deploy via a GitHub workflow for Kanto

Dear Eclipse Team,

We currently have Kanto's website built using Hugo and we want to automate the process using a GitHub workflow.

We are aware that we can request a Jenkins job to be configured for that purpose but we really want to try to keep everything as open as possible and have it consistently available for updates and configurations in the project's main repository.

Looking into the provided Jenkins Job example, we can break it down into a couple of steps:

• checkout
• build
• push the diff to the dedicated repo (and branch) via an authorized (SSH) machine user

As a step one - we do not foresee staging and want to keep it simple to begin with small steps.

What we would imagine is doing the same via GitHub actions in a workflow utilizing the GitHub Deployment Keys (or a token if possible) that can be configured and is accessible securely within a workflow for the machine user to authorize from the GitHub repo to the target one where the built web resources need to be pushed.

@webmaster

Does that sound viable? Do you think you could support us on this one?

Any ideas and expertise are much appreciated and we will be happy to have your opinion and help on this one!

changed title from Hugo documentation build and deploy via a GitHub workflow for Kanto to Hugo website build and deploy via a GitHub workflow for Kanto

We are aware that we can request a Jenkins job to be configured for that purpose but we really want to try to keep everything as open as possible and have it consistently available for updates and configurations in the project's main repository.

This sounds like the workflow of updating a project's website through a Jenkins job would be less open than a GitHub action workflow. That's not the case.

As mentioned in the Jenkins wiki, the project's website (Hugo) source code can be developed in a GitHub repo. Even the Jenkins pipeline (in a Jenkinsfile) can be store there. The Jenkins job only pushes the "compiled" website to the www-repo.

Does that sound viable? Do you think you could support us on this one?

For now, we only support pushing to a project's website repo (git.eclipse.org/www.eclipse.org/kanto in this case) via Jenkins. We are planning to support other ways of handling project websites, but there is no timeline for it yet.

@fgurr Thank you for your quick response!

This sounds like the workflow of updating a project's website through a Jenkins job would be less open than a GitHub action workflow. That's not the case.

You are absolutely right - poor wording choice - sorry!

Our main goal is mainly to try to keep the tech approach (used infra, tooling, etc.) as simple and as consolidated as possible - i.e. using the same tooling (if possible) for all the tasks we need to fulfill.

Uploading the web resources in the end boils down to a git push via a SSH-enabled machine user. This is possible to achieve using the mentioned repo's Deployment Keys (securely stored, accessible and manageable by the Web Masters team only) and should not interfere with the current Jenkins approach or require a new API or else.

Is there a technical obstacle we would not like to give this approach a go?

Is there a technical obstacle we would not like to give this approach a go?

We have no automatic process in place to set and update deployment keys in GitHub. Hence this is unsupported for now. Priority will raise with raising demand for this feature.

As part of working towards moving project websites into the eclipseprojects.io domain we're going to do some work on our tooling to support checking out the website content from repos hosted on github.com/eclipse* and gitlab.eclipse.org rather than forcing everything to be in our existing repos.

This would allow a potential solution of building the static site with Githubs tools and then we could check out the 'build results' branch periodically.

This doesn't resolve the 'make a change and have it go live immediately' situation, but it give the project a bit more flexibility.

@fgurr

Got it! Thank you for your patient and thorough answers!

@mward

Thanks for jumping in with this info - sounds promising How can we track the progress of this migration and tooling improvements? (Sorry if I missed a notification e-mail or a post on the topic)

In the meantime, I guess we go with Jenkins until we have an alternative - I'll post a dedicated ticket for requesting an instance for the project :)

Right now it's somewhere between theory and highly experimental(#732 (closed)), and the timing is best described as 'in the near future'.

We are working on a more concrete plan for the website transition, and we'll share that with the community once it's ready.

mentioned in issue #1425 (closed)

Hey @mward!

As promised - pinging here to continue the transition discussion from #1425 (closed)

• The designated branch in the eclipse-kanto/kanto repo is website
• The expected website's base URL is updated accordingly to https://www.websites.eclipseprojects.io/kanto/
• The dedicated GitHub workflow to automate the Hugo build and push to the designated branch is up and running too

One question on my side - the redirects from eclipse.org/kanto to websites.eclipseprojects.io/kanto will be enabled - right? I just want to make sure we do not have an "offline" moment and disturb the Kanto explorers : ))

Ok I've updated the PMI details for Kanto to set the correct Github repo and branch, and the prior content has been removed(from the 'staging' server), and the update process has pulled in the new site. So far so good.

The actual final url will be https://websites.eclipseprojects.io/kanto , no www 'prefix'.

Currently there are no redirects enabled, but I have a patch up for review which will do so, but it's meant add the redirects for all impacted projects at once. If you're happy to use the new site, I can add a specific redirect for kanto, or you can update the www.eclipse.org/kanto.git repo and put a redirect in the index.php file pointing to the new site if you want more control over the 'when'.

OK - noted and fixed the prefixing issue with a PR just now. The website's content with the new base URL is built and pushed to the branch.

I guess that when the tooling pulls it again - we'll be able to see the website live at https://websites.eclipseprejects.io/kanto, right?

Once it gets online, we'll proof-click and proof-check it and will confirm the redirect you offered in the index.php

Thanks again for your immediate support!

I guess that when the tooling pulls it again - we'll be able to see the website live at https://websites.eclipseprojects.io/kanto, right?

Yup, the 'sync' time is roughly every 5 minutes.

Thanks again for your immediate support!

NP.

Hmm.... I hope I'm just too impatient : ))

Try: https://websites.eclipseprojects.io/kanto rather than https://websites.eclipseprejects.io/kanto

I see that I mistyped that earlier so I'll fix those comments.

Sorry! Yet another solid proof for "no work in the late night hours!"

All live now!

At first glance, what I can share is that the newest built content is missing - I see it in the branch - properly generated and linked but I do not see it added into the live version for some reason.

Such an example is the content in this dir - https://github.com/eclipse-kanto/kanto/tree/website/docs/concepts that is expected to be visible alongside the >Getting started node in the Documentation page left-hand side menu

It also looks like the certificate used by the website is not valid for domain websites.eclipseprojects.io but only for eclipseprojects.io which causes a warning/error in the browser when opening it ...

At first glance, what I can share is that the newest built content is missing - I >see it in the branch - properly generated and linked but I do not see it added >into the live version for some reason.

It's a good thing we're testing this!

So the bug in this case is that the update tooling doesn't have any logic to handle changes in repo(or branch), as I presumed that those things wouldn't change very often(if at all). In this case even though I removed the 'old' content there was clearly some caching of my API queries so the original git.eclipse.org repo was re-used before it changed to Github. I've fixed that on the live site, and I'll put a code patch together.

It also looks like the certificate used by the website is not valid for domain

This was on my list of things to fix.

It's a good thing we're testing this!

Yup! We'll push a lot of new content in the upcoming days so we'll have the chance to confirm the fix & patch are stable - keep you posted

This was on my list of things to fix.

Thanks for fixing this one as well!

I've pushed the patch live, so let's see how it goes.

Hey Matt,

it looks like we have the same sync issue again - new content is available at https://github.com/eclipse-kanto/kanto/tree/website/docs/how-to-guides and unfortunately it doesn't get displayed on https://websites.eclipseprojects.io/kanto/docs/ under a new How-to guides section.

Any thoughts?

Please note, for the next few weeks webmasters will operate with reduced staff due to the vacation season, so responses may be delayed.

The update container appears to have crashed and is not being restarted cleanly. I'll take a look in a week or so when I'm back.

Thank you @mward ! Have a great off-work time

@mward the website content is being properly live updated - thanks for your and your team's help!

Overall, we have tested the new live update process and infra to achieve it - no show-stoppers found - works as expected so far. As we have piled up a lot of new content on the new domain while testing and have not found any functional problems with this approach, we would like to enable the redirects for Kanto's official website - from https://www.eclipse.org/kanto/ to https://websites.eclipseprojects.io/kanto/ - so that we make this content easily available for our contributors

Thanks for your continuous support on this topic!

@kgramatova thank you for the feedback. Matt is back next week, can he look at this then, or do you need anything sooner?

@droy Thank you for getting back on this one so quickly! It's always best to have things moving fast when it comes to user doc and enabling our potential contributors/adopters, but I guess a weekend to wait won't hurt as long as we have it ready early next week

I hope this would be feasible on your team's side as well

assigned to @mward

changed due date to August 16, 2022

I've set a reminder for Matt for next week. Thanks

I've submitted a change to the www.eclipse.org configs to redirect /kanto* to websites.eclipseprojects.io/kanto* and it should go live around midnight tonight.

I also checked in on the previous issues with the update container and it looks like everything has gone back to normal.

Thanks @mward and team!

Confirming - everything works as expected

I think we can close this ticket now as the general infra & domain migration and redirects look successful - if any issues come up after, we'll post them in a dedicated ticket.

Great, thanks for following up, and for volunteering your project as a test subject for the new system.

closed

mentioned in issue #2040 (closed)

mentioned in issue #2070 (closed)