Proposal: deprecate PHP for project websites

added Security ~40426 statetodo team:infra team:webdev labels

assigned to @droy

what effect does this have to downloads pages like https://www.eclipse.org/modeling/tmf/downloads/ (where i even dont know where the sources are) will there be a generic replacement?

@cdietrich Can you explain what that page does in order to display that information? Are you reading files from a specific directory or are you fetching data from our databases?

i dont know. all was there from before my committer time. i assume the code was inherited from emf. as i said i dont even know where the php files are as sourcecode

Assuming that you are reading the files: You won't be able to do that anymore with PHP. However, I assume you would have access to that directory via your Jenkins instance.

If so, you could replace all the php logic with a cron job that scans the filesystem and generates a JSON file with all the data you need to generate your download page. You would then need to fetch the data using JavaScript and print a list of downloads.

With that said, is that worth the effort? I am currently looking at the Google Analytics stats for that page and there was only 3 pageviews for that page in the last week.

what would the download page display if there was no php. e.g. is dir listing enabled. i currently just use it to find out the download link to a specific version zip how does one construct the normal download links like https://www.eclipse.org/modeling/download.php?file=/modeling/tmf/xtext/downloads/drops/2.28.0/R202208290555/tmf-xtext-Update-2.28.0.zip how can if find out that path

Are you asking what would people see if we disable php and the website php code stays on the webserver?

or if the php code is deleted.

the main goal is: how can i and users find out the correct download links to the zips e.g https://www.eclipse.org/modeling/download.php?file=/modeling/tmf/xtext/downloads/drops/2.28.0/R202208290555/tmf-xtext-Update-2.28.0.zip

do you have stats for https://www.eclipse.org/modeling/download.php..... too

looks like directory listing is at the actual download place https://download.eclipse.org/modeling/tmf/xtext/downloads/drops/

This does not need to be complicated. On those PHP index pages that never change, just save the HTML and commit it as index.html and remove the index.php. 2 minutes of work.

this page changes with each nightly and milestone/releases that is deployed

As mentioned, run the PHP script from Jenkins after build artifacts are published, and commit the html output to www.eclipse.org/modeling/tmf/downloads/index.html

Or better yet, redirect it to the index on download.eclipse.org . You have options.

doing bash scripting with jenkins with no backup and testing possibility ....

... is better than a legacy, orphaned PHP script that is abandoned and no longer maintained.

if you dont have no time its not. then every change to a running system is bad

This is an abandoned running system on a production system. It's bad from the start, and status quo is unacceptable. Sorry.

(as an aside, it would be more productive to channel your energy towards community outreach and seek assistance. I've done that with the Babel project and recruited two new committers. Or, the project will be up for archival as I have no time to work on it. I am in the same boat as you are. #1749 (closed))

all attempts todo so over the last 3 years failed. also doing "releng" is the worst part of oss that gives fun to nobody so recruiting somebody to "adapt stuff that works and no longer will cause infrastructure changes" is not an good marketing attempt. so one day i will give up and pull Xtext from Simrel and all that is below it.

@cdietrich Let's suppose that we decide to keep supporting PHP for the project website. There is a strong possibility that your project website won't work in PHP 8: https://www.php.net/supported-versions.php

PHP 7 security support ends in 1 month.

From our perspective, if a project decides to build a website for their project, we expect them to maintain it as we do need to apply security updates to our servers by upgrading various software such as but not limited to PHP. If the project does not have the resources to maintain a website, we strongly recommend that they use the PMI as their project website: https://projects.eclipse.org/projects/modeling.tmf

I do believe that if you chose to convert your PHP site to static HTML, you have a solution that can last way longer than any PHP solution.

Also, by leveraging Jenkins to generate your content, you can choose to write your tool in any language that you are comfortable with.

If you prefer Java, I am sure you could migrate that PHP functionality to Java which will make it easier for your project to maintain since not everyone is a PHP developer.

We are making this recommendation to allow us to modernize our infrastructure and improve security for us and our users.

We understand that this is a challenging problem for us and our projects since we've been running some of these websites for more than a decade. This is why we are announcing this proposal early. This is not something we will impose to our project this year but it's definitely in the direction that we think is best for all (users, foundation and our projects).

From my perspective, I don't believe it's reasonable for us to run/support server-side code that is no longer being maintained on eclipse.org. I think the risk to our users is too high.

the problem is the "short notice". i am supposed to care about

normal work for 2022-12 release
deal with php deprecation
deal with website changes
deal with the forced xtend website changes all in 1-2 month

deal with php deprecation

At this time, this is only a proposal. We decided to create an issue early to communicate our intentions early with the community.

I can guarantee that you don't need to worry about this in 2022. I can also assure you that will give more than a few months to our project to deal with this.

From your perspective, let's say that we decide to move forward with this proposal, what would be a reasonable timeframe between the official annoncement to when PHP support would be officially removed from our servers?

6 months, 1 year, or more than that?

i have now updated the docs. so the problem still is backwards compatibility. there are tons of links like https://www.eclipse.org/modeling/download.php?file=/modeling/tmf/xtext/downloads/drops/2.28.0/R202208290555/tmf-xtext-Update-2.28.0.zip or https://www.eclipse.org/modeling/download.php?file=/modeling/emft/mwe/downloads/drops/2.13.0/R202205191115/emft-mwe-2-lang-Update-2.13.0.zip out there. please note: this download.php seems to be owned by the modeling toplevel project, and be used by unknown number of subprojects

i wonder if a redirect for these urls can be done

This is the source code of https://www.eclipse.org/modeling/download.php

Your page basically redirects the user via javascript to the foundation downloads script:

https://www.eclipse.org/modeling/download.php?file=/modeling/tmf/xtext/downloads/drops/2.28.0/R202208290555/tmf-xtext-Update-2.28.0.zip

redirects to

https://www.eclipse.org/downloads/download.php?file=/modeling/tmf/xtext/downloads/drops/2.28.0/R202208290555/tmf-xtext-Update-2.28.0.zip

To support you, we would be more than happy to add a redirect rule that ensures that these links continue to work in perpetually. We would simply redirect all traffic from https://www.eclipse.org/modeling/download.php?file=* to https://www.eclipse.org/downloads/download.php?file=*

This would be a small improvement to what you currently have since your current redirect only works if the client making the request is running javascript.

You can ask us to do so by creating an issue in our helpdesk.

But the latter php would also be gone if php is gone. @emerks who owns the modeling/download.php

But the latter php would also be gone if php is gone.

Are you referring to https://www.eclipse.org/downloads/download.php? If so, that's not a project website. Everything under /downloads/ is owned and managed by the Eclipse Foundation. The IT team, which I am a part of, is responsible for maintaining that code.

My team is currently migrating most of eclipse.org content owned by the foundation to static HTML. You can follow that work via this project: https://gitlab.eclipse.org/eclipsefdn/it/websites/eclipse.org

For example, the eclipse.org homepage is no longer being served by PHP. This has been true for a few months, it's now a static HTML page generated by Hugo.

However, the download service is not something we can migrate to static HTML. If we find a way, we will ensure that we don't break any existing functionality. If we replace .PHP in the download section, we will ensure that we have redirects in place to the new service since I suspect these links exist all over the internet and we definitely don't want to break them.

The foundation does need to continue to support PHP for some services that the foundation maintains such as but not limited to eclipse.org/downloads, accounts.eclipse.org, marketplace.eclipse.org, projects.eclipse.org, api.eclipse.org, and eclipsecon.org.

One of the responsibilities of my team is applying security updates to each of these drupal/PHP applications each time they come out.

However, we've been working really hard in the last few years to reduce our PHP footprint to reduce complexity and the maintenance burden of keeping up with all the security patches. We used to build all our WG websites using Drupal however, a few years ago we decided to prioritize Hugo which is a static website framework.

Are you referring to https://www.eclipse.org/downloads/download.php? If so, that's not a project website. Everything under /downloads/ is owned and managed by the Eclipse Foundation. The IT team, which I am a part of, is responsible for maintaining that code.

@cguindon does this mean that the EF will make sure that URLs starting with https://www.eclipse.org/downloads/download.php? will continue to work and that projects using such URLs for letting users download artifacts will not need to change them?

@khudalla absolutely 100% no change there. We are deprecating the usage of PHP for Eclipse project web content. https://www.eclipse.org/downloads/ is an EF property that is not affected by this deprecation at all.

mentioned in issue #1776 (closed)

I assume(d) that PHP in general will/would not be "gone":

#732 (comment 1013220)

But maybe it's a bad assumption to assume that things that work should just continue to work. No one said boo or bah on the above thread, but now there is this thread in parallel. I'm not sure how should I discover that this thread exits. Might I expect to be made aware of this "proposal" in some way other than through indirection, weeks later?

I should point that some of "us" and probably many of "us" do not have a team of people to do work to make things that worked before work again in some new and improved way. In fact some of use have a hard time keeping up with the flow of email traffic on a daily basis.

Of course I assume that only the best of intentions underlie all these actions and that the "team" will help us all on the path to the new and better way.

sigh you guys really need to re-read the description of this issue.

Seriously.

Sorry Denis, I know you said you'd work with us. Speaking for myself, it is really super hard these days to keep up with all the disruptive changes, not just from the infrastructure, but also from upstream projects, so now we are overwhelmed and a bit paranoid. I know @cdietrich is generally running a one man show too and so much depends on him delivering timely quality results.

Sorry, sorry. It's just frustration from "too much to do". No offense is intended.

Totally get that. We've learned from our past mistakes, so this issue is "pre-communication". There is nothing to do yet.

There is no plan right now, and no immediate action item. Authored here is the problem statement and a proposed solution. The EF will work on communicating this issue, gathering feedback and working with a community on a plan to resolve the problem while focusing on minimizing disruptions to the community.

(in other words, before communicating this, we want to wait for the dust to settle on other issues as we agree it's a lot. For now, we're simply tracking the problem statement)

mentioned in issue #3652 (closed)

We've identified three main areas where PHP code, running on EF servers, will be affected by this proposal:

Project websites on https://eclipse.dev/ These are likely stable/mature projects that use the then-provided PHP templates for otherwise static websites. Some projects use PHP to craft download pages or other dynamic content. Some very old pages use database libraries to query the bugzilla database.
Project download pages on https://download.eclipse.org/ and/or https://archive.eclipse.org/ PHP is mainly used here to read the file/directory contents on the filesystem, and produce index and helper pages for downloads.
Community-created and EF-backed initiatives, such as those for maintaining Eclipse MarketPlace client, Eclipse Simrel and other such initiatives.

Here is what the team is proposing:

PHP on project websites on eclipse.dev: projects must migrate to static content by Dec 31, 2024 (in 10 months). The EF's WebDev team will provide migration paths to assist.
PHP used for download/archive pages: projects must migrate to static content by Dec 31, 2025. (in 22 months). The EF's WebDev and Releng teams will provide a sample Jenkins job that can be used to provide a static webpage based on directory contents.
PHP in EF-backed initiatives: These would need to be considered individually. In most cases, PHP would remain; these properties would therefore be owned and managed by the EF's WebDev team to ensure a consistent application of a code review/deployment processes across all the code that runs on EF websites. All current authors would be expected to follow this process, which we consider simple in nature, unobtrusive and standard practice for software development. Regardless of the case, we expect these transitions to be complete by Dec 31, 2025.

We realize that the EF has many items in a deprecation state - either in progress, or as is the case here, proposed and in planning. This is the unfortunate consequence of maintaining an infrastructure that has not properly evolved as the environment around us has changed, and even in yesteryear's security context, are no longer fit or are simply unacceptable.

The EF IT team will create an overall deprecation document to allow everyone to see, at a glance, the items in deprecated state, as well as the status, timelines and the migration paths. Edit: the document is here: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Eclipse-Service/Infrastructure-changes-Guide

When can we expect information on the migration paths for the websites? The deprecation document currently does not contain this information.

The @webdev team is looking into that. In the meanwhile, you can look at the Hugo Solstice theme - most projects start there:

https://gitlab.eclipse.org/eclipsefdn/it/webdev/hugo-solstice-theme

There is also a boilerplate project that you can look at: https://gitlab.eclipse.org/eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate#how-to-build-my-projects-website-with-jenkins

It's a great starting point with documentation and it include an example of what to include in your Jenkinsfile: https://gitlab.eclipse.org/eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate/-/blob/main/Jenkinsfile?ref_type=heads

I had a quick look on hugo. For us this seems a very nice replacement of our php based page. Just for clarification: for project that is on github we would have to github repositories for the project web-page:

project web-page hugo source
github repo web-page that is then mirrored to eclipes.dev

Am I right? Could 1 be a branch of 2?

The publishing tool behind eclipse.dev does have a rudimentary understanding of branches, so yes it's possible to have a single website repository at Github(or this Gitlab for that matter), with 2 branches(eg: source and publish) where a CI job checks out the 'source' branch, runs the Hugo build and commits the results to the 'publish' branch in the same repo, and the eclipse.dev publishing tool would pull from the 'publish' branch.

@mward would you recommend this practice, or split it to two repos? We are on github if this matters.

The location of the repo(Gitlab v. Github) doesn't really matter, I was just trying to be clear that the publishing tool was 'repo location neutral'(provided those locations are this Gitlab instance or Github).

As for multiple branches vs. multiple repos, I don't really have a strong opinion(although as an old school sysop I lean in the direction of multiple repos).

When we started moving project website repos away from git.eclipse.org we tended to go with separate repos mostly for reasons of history and the older tooling didn't support branches.

The transition to eclipse.dev brought branch support which makes it possible to 'do it all' in a single repo now.

Which is a long way of saying: it up to the projects to determine which of those options is the 'best'(however you choose to define it) for their projects.

Thx for the detailed explanation. After playing a bit with hugo and the boilerplate project I also think the separate repository approach is the better. I would find it weird to have a git repo where in two branches have a completely different content.

I too have just discovered this thread.

The scripts in question were authored by Nick Boldt and from the perspective of committers were imposed on containerised projects by the EF. They provided a uniform configurable solution to problems so that the many containerised projects exploited the efforts of one releng.

Sadly the EF no longer recognises the value of good common solution from one releng at the EF helping all. Instead each project is forced to develop its own solutions.

I recognise that bit rot has set in wrt PHP so we need to move on. I have made a couple of requests for guidance on how the EMF / Orbit Download pages are produced but received no response. I have attempted to study the EMF / Orbit builds and am bit surprised / embarrassed that I cannot even figure out whether there are pre- or live-computed pages; the traditional single step with a debugger is hard when I cannot even identify what to single step.

Since PHP has many problems, I am looking for 'boring' Java code that computes the pages at build time so that I can review and debug on my local machine. IMHO the EF must provide examples of such facilities with guidance on how they can be tailored.

Until the EF provides support for the replacement for the EF promoted tooling, I am unable to update the longstanding PHP for my projects. It still works after a fashion. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=436605

Sadly the EF no longer recognises the value of good common solution from one releng at the EF helping all. Instead each project is forced to develop its own solutions.

The comment just abouve yours provides exactly that.

Since PHP has many problems, I am looking for 'boring' Java code that computes the pages at build time so that I can review and debug on my local machine. IMHO the EF must provide examples of such facilities with guidance on how they can be tailored.

Good idea; I've filed the issue below for just that. No guarantee it will be "boring java code" but it will be something that will produce the expected result:

https://github.com/eclipse-cbi/jiro/issues/347

I've just started with JustJ; looks very promising, but definitely needs a for-migrating-newbies example with an enumeration of mandatory / optional customisations. It seems to only work if packaged in a follow-on plugin. When confident I'll report issues against JustJ.

(Surely this Update Site generation is out-of-scope wrt JustJ's JRE support? I searched for it a few months ago in EMF and OOMPH.)

On past cross-projects posts I have mentioned that the following two URLs are very useful:

Those URLs allow one to explore the direct contents of the servers so that one can determine how the content is represented.

I have also drawn attention to the following documentation for the reusable infrastructure for generating and managing update sites:

https://eclipse.dev/justj/?page=tools

I try to keep my posts on cross-projects short, because it's evident that very few people read the content...

Ironically all above links use PHP and the content is just 4 years old. Re-authoring it is not high on my list of cool things to do. I don't think the first two links are replaceable with something other than PHP...

Thanks just what I wanted.

I do read your cross-project-dev posts; they are usually good value.

Unfortunately I blind-spotted justj since I thought it was solely concerned with shipping an integrated JRE.

@emerks there are cases where PHP is useful and justified, and in the case of the JustJ pages, I would like to propose that they fall under category "3. PHP in EF-backed initiatives" and would like to suggest that "these properties would therefore be owned and managed by the EF's WebDev team to ensure a consistent application of a code review/deployment processes across all the code that runs on EF websites"

Provide sample build job that produces a download page for download.eclipse.org: https://github.com/eclipse-cbi/jiro/issues/347

Looks like PHP was take down on eclipse.dev already. eek!

This https://eclipse.dev/jetty/documentation.php now results in an "Application not available".

The Jetty project is in the process of migrating away from php, and will be done this summer, well before the proposed date of Dec 31, 2024.

Looks like PHP was take down on eclipse.dev already. eek!

This https://eclipse.dev/jetty/documentation.php now results in an "Application not available".

This was an unscheduled brown out. Site is back now.

The "Eclipse" Project also uses some minimal PHP for it's "New and Noteworthy" pages. See e.g. https://eclipse.dev/eclipse/news/4.31/platform.php

We discuss here: https://github.com/eclipse-platform/www.eclipse.org-eclipse/issues/122#issuecomment-1947881567 how to move away from that.

See the questions in https://eclipse.dev/eclipse/news/4.31/platform.php. Can you pls. give us guidance how to migrate that?

Can you pls. give us guidance how to migrate that?

As mentioned, we'll provide some migration guides shortly. We'll try to strawman something next week. Perhaps this comment from @cguindon can provide some initial insight? (ie, the PHP page would be replaced with a built html from Jenkins)

#1934 (comment 1750003)

As we get ready to close the proposal phase of this issue and declare PHP actually deprecated, @cguindon and @fgurr could your teams please craft/aggregate migrations path documentation for:

project PHP website content on eclipse.dev
project PHP download content pages that typically run on download/archive.e.o (ie, how to leverage CI to

into a single page that we can reference when officially announcing deprecation?

Thanks!

Hi @droy,

We have this hugo-eclipsefdn-website-boilerplate project which includes instructions/documentation in the readme: https://gitlab.eclipse.org/eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate/-/blob/main/README.md?ref_type=heads

I suggest Fred and I take some time to review both the content and the example Jenkinsfile to ensure everything is current.

Would this work for you?

Absolutely. Please let me know once it's reviewed, I'll then use the 15+ year old Babel project website as a guinea pig for migrating off PHP for project websites, for which the source is found at https://github.com/eclipse-babel/babel-website . I haven't done anything productive with it in over a decade.

Perfect - This task is in now in my queue and I plan to look into this on next Tuesday afternoon. I will report back once I am done!

I made some progress on this task this afternoon.

I created a new MR with a draft for updating our documentation. I've asked a few colleagues for feedback (however, anyone following this MR is welcome to post a comment): eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate!7 (merged)
I also created an MR for the Eclipse CBI project to create a new Docker image for Hugo 0.110.0: https://github.com/eclipse-cbi/dockerfiles/pull/36

Hugo 0.110.0 is currently the official version that our Hugo Solstice Theme supports.

Updates:

I made some edits to the documentation yesterday. Here's a direct link in case anyone is interested in providing feedback: https://gitlab.eclipse.org/eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate/-/blob/b3011ac46fda03522ae61d576ca4c385388021cc/README.md

With these edits, I also decided to create a new issue template in the Helpdesk to help streamline the creation of project website support requests by asking projects to provide information that will help us help them.

The new issue template is now available: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/new?issuable_template=project-website

Link to MR: eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate!7 (merged)

My MR was merged today.

I also made a separate MR to update the boilerplate to utilize our new look and feel called Astro: eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate!9 (merged)

@fgurr do you plan to make any changes to the Jenkinsfile or is it still good as-is? https://gitlab.eclipse.org/eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate/-/blob/main/Jenkinsfile?ref_type=heads

I'm planning to make changes. The template for GitLab and GitHub repos is slightly different, so I'm not sure if it's better to have two different Jenkinsfile templates.

We should also add a template for a GitHub action, and a link to a GitLab CI template.

I'm planning to make changes. The template for GitLab and GitHub repos is slightly different, so I'm not sure if it's better to have two different Jenkinsfile templates.

Awesome! An idea would be to create a Jenkinsfile.gitlab and Jenkinsfile.github file in the root and add some instructions in our docs to rename the one they need to Jenkinsfile.

Done => eclipsefdn/it/webdev/hugo-eclipsefdn-website-boilerplate@184d8548

The Jenkinsfile can either be renamed or specified explicitly (the name of the Jenkinsfile that should be used can be set in the Jenkins job config).

assigned to @cguindon

mentioned in issue eclipse-wg/ide-wg/ide-wg-dev-funded-efforts/ide-wg-dev-funded-program-planning-council-top-issues#15

Proposal: deprecate PHP for project websites

Designs

Child items ...

Activity