eclipsefdn-api-common issueshttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues2024-03-11T15:57:57Zhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/102Project collaborations without names will break Project preloading2024-03-11T15:57:57ZMartin Lowemartin.lowe@eclipse-foundation.orgProject collaborations without names will break Project preloadingDiscovered through the Project Adopters API, if a project has a collaboration set without a name, it will cause the project loading to break and may cause cascading failures through the API.Discovered through the Project Adopters API, if a project has a collaboration set without a name, it will cause the project loading to break and may cause cascading failures through the API.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/101Catch Hibernate exception for empty results in framework2024-03-05T15:54:19ZMartin Lowemartin.lowe@eclipse-foundation.orgCatch Hibernate exception for empty results in frameworkAt some point in the Quarkus 3.x upgrade, the platform switched from returning empty lists to throwing runtime exceptions when there are no results for SQL queries. We already handle checking as needed, so to go back to previous state an...At some point in the Quarkus 3.x upgrade, the platform switched from returning empty lists to throwing runtime exceptions when there are no results for SQL queries. We already handle checking as needed, so to go back to previous state and handle this a little more cleanly, a patch will need to be made to return an empty list in these cases.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/100Quarkus 3.8 LTS upgrade2024-03-20T14:17:36ZMartin Lowemartin.lowe@eclipse-foundation.orgQuarkus 3.8 LTS upgradeIn early July, Quarkus 3.2 will no longer recieve security updates as it will reach it's 1 year end of life. At that point, we will need to migrate to the new LTS of Quarkus 3.8. There is currently one breaking change that may impact us,...In early July, Quarkus 3.2 will no longer recieve security updates as it will reach it's 1 year end of life. At that point, we will need to migrate to the new LTS of Quarkus 3.8. There is currently one breaking change that may impact us, though investigation will be needed to check the scope of the impact. The migration guides and one of the guides mentions that anything below the announced minimums will cause the API to fail to start (https://github.com/quarkusio/quarkus/wiki/Migration-Guide-3.5#hibernate-orm), though it is mentioned if you explicitly set the version that older versions may start though they could experience issues.
The only DB that should be on the old MariaDB version is one of our internal API DBs that isn't exposed to the public, so we might be able to branch our release and do fix backports in the short term in the worst case scenario.2024-06-07https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/99Add eager token refresh to token service2024-02-28T16:15:46ZZachary SabourinAdd eager token refresh to token serviceWhile monitoring the Profile-API, multiple unauthenticated requests were made to accounts at the same time using an expired token. This caused the required `mail` field to be missing, and caused the requests to fail.
We should make the ...While monitoring the Profile-API, multiple unauthenticated requests were made to accounts at the same time using an expired token. This caused the required `mail` field to be missing, and caused the requests to fail.
We should make the token refresh happen shortly before the token is set to expire to avoid this ever happening.Zachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/98Add cleanup task for distributed CSRF table2024-03-20T14:17:36ZMartin Lowemartin.lowe@eclipse-foundation.orgAdd cleanup task for distributed CSRF tableThe current persistence runtime adds support for distributed CSRF but doesn't account for the cleanup of the table. Since this is user data, we should clean up any entries older than an hour on some regular schedule to ensure we don't re...The current persistence runtime adds support for distributed CSRF but doesn't account for the cleanup of the table. Since this is user data, we should clean up any entries older than an hour on some regular schedule to ensure we don't retain private data longer than necessary.https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/97Add alternate to caching layers to use external/distributed caching2024-03-20T14:17:36ZMartin Lowemartin.lowe@eclipse-foundation.orgAdd alternate to caching layers to use external/distributed cachingCurrently, we use an in-memory cache for caches for our services. As we continue to integrate with external services and scale up the size and scale of our pods, the need for a distributed cache increases. To better account for these cas...Currently, we use an in-memory cache for caches for our services. As we continue to integrate with external services and scale up the size and scale of our pods, the need for a distributed cache increases. To better account for these cases, we should look to using a distributed cache technology like Redis to supplement our current in-memory caching strategies. Ideally, we would be able to switch standard and loading caches independently to use Redis or the in-memory Caffeine cache as needed for applications to give greater flexibility when handling expensive to calculate values and faster quick/inexpensive to calculate values.
We currently don't have access to a Redis instance to develop against, so that would need to be taken care of first before we could begin the implementation of this feature.https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/96Update mail_history field in EfUser model2024-02-02T15:13:04ZZachary SabourinUpdate mail_history field in EfUser modelAn error was made and the `mail_history` field is not in the proper format. It needs to be updatedAn error was made and the `mail_history` field is not in the proper format. It needs to be updatedZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/95Update template header logo to 150px width2024-01-08T21:16:03ZZachary SabourinUpdate template header logo to 150px widthZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/93Set maximum default cache sizes to avoid cache OOMing the service2024-01-02T15:55:09ZMartin Lowemartin.lowe@eclipse-foundation.orgSet maximum default cache sizes to avoid cache OOMing the serviceWhile testing profile, Zach and I noticed that while initial cache capacity is set, we never set a maximum. This could cause issues where the cache completely fills the containers, or grows to an unmanageable size. To combat this, we sho...While testing profile, Zach and I noticed that while initial cache capacity is set, we never set a maximum. This could cause issues where the cache completely fills the containers, or grows to an unmanageable size. To combat this, we should set a default limit of 10k entries, which should be enough for most of our use cases. Any other cases, we can just override those build properties in the downstream applications easily.Zachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/92Properly map Runtime Exceptions to 500 returns2023-11-27T14:51:25ZZachary SabourinProperly map Runtime Exceptions to 500 returnsOur [RuntimeMapper](https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/blob/main/core/src/main/java/org/eclipsefoundation/core/resource/mapper/RuntimeMapper.java?ref_type=heads) currently maps RuntimeExceptions to `400 ...Our [RuntimeMapper](https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/blob/main/core/src/main/java/org/eclipsefoundation/core/resource/mapper/RuntimeMapper.java?ref_type=heads) currently maps RuntimeExceptions to `400 Bad Request`. They should be mapped to `500 Internal Server Error`.
A call to foundationdb-api earlier today received a 400 after a DB error occured. It was not immediately clear from the error what the problem was. A 500 should make this easier to understandZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/91Update caching service to allow proper distinction between error states2023-12-05T15:29:32ZZachary SabourinUpdate caching service to allow proper distinction between error states### Problem
The current implementation of our caching layer has a few holes that need to be addressed.
If an error occurs while caching data such as LDAP responses, HTTP responses, or large data processes, the `CachingService`'s current...### Problem
The current implementation of our caching layer has a few holes that need to be addressed.
If an error occurs while caching data such as LDAP responses, HTTP responses, or large data processes, the `CachingService`'s current behavior is to log the stack trace, suppress the error, and return an empty `Optional`. This solution works well for some cases, but is insufficient when it comes to passing a meaningful error to the client.
**EXAMPLE:** The RESTEasy client throws a `ResteasyWebApplicationException` for both 404 and 500s. This causes an error while creating a cache key, but the cache only returns an empty `Optional`. In the case of fetching a user from Accounts or fetching from Foundationdb-api, this makes it impossible to determine the difference between data not existing and the external service being down without looking at the logs. Seeing as we generally assume an empty `Optional` means the data doesn't exist, the final return to the client is always a 404.
In the case of the sync script, this would lead to user permissions being removed if LDAP or Accounts were experiencing some service disruptions.
### Proposed Solution
Create a cache wrapper object with some relevant fields:
- An `Optional` containing the cached data
- An `Optional` containing the error if there is one
- We can ad additional fields that we deem relevant
Instead of returning and `Optional`, the caching service would return this object. Also in the case of an error, we could allow for a refresh if the error is not a 404.
/cc @maloweZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/90Investigate adding SQL comments to persistence queries for logging2024-03-20T14:17:35ZMartin Lowemartin.lowe@eclipse-foundation.orgInvestigate adding SQL comments to persistence queries for loggingIn SQL, comments can be included with queries to provide additional context to the call in the DBs event logs. For the most part, Hibernate (our DB engine) doesn't seem to support comments very well, and Quarkus might not have added the ...In SQL, comments can be included with queries to provide additional context to the call in the DBs event logs. For the most part, Hibernate (our DB engine) doesn't seem to support comments very well, and Quarkus might not have added the option to configure this setting. We will need to do some internal testing to see if adding HQL comments to queries will result in proper comments in the SQL log.https://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/89Add regex helper to handle backtracking attacks2023-11-08T20:14:30ZMartin Lowemartin.lowe@eclipse-foundation.orgAdd regex helper to handle backtracking attacksCurrently, there are a few regexes present in our code that could technically lead to a specifically formulated string to attack our services. The risk of this is somewhat low, but we should address this by adding timeouts and limits to ...Currently, there are a few regexes present in our code that could technically lead to a specifically formulated string to attack our services. The risk of this is somewhat low, but we should address this by adding timeouts and limits to better handle these cases.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/88Update WGService to include additional features and filtering2023-11-09T14:27:33ZZachary SabourinUpdate WGService to include additional features and filteringZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/87Add toggleable hostname flag to HTTP responses2023-11-08T16:03:04ZMartin Lowemartin.lowe@eclipse-foundation.orgAdd toggleable hostname flag to HTTP responsesTo better support multi-pod deployments, we should add a call to our response chain that adds the pod's hostname to a response header when enabled (by default this should be disabled). This can be done through a call to the host machine,...To better support multi-pod deployments, we should add a call to our response chain that adds the pod's hostname to a response header when enabled (by default this should be disabled). This can be done through a call to the host machine, with the `exec` answer from this [SO answer](https://stackoverflow.com/a/28043703) being the most stable mechanism in our case. While this won't work on Windows, that doesn't really matter since the only time this will matter is in Unix environments.
Regarding the command to use, it looks like in our pods the safest command is likely `uname -n` rather than `hostname` as not all unix instances have access to the `hostname` shortcut. We should cache this once and reference and reuse the cached answer as this will never change in the runtime of a server, and we should reduce native system calls where possible.
I think a good name for the header could be `x-node-name`, as the value we're pulling is the node hostname of the server rather than an ID of any kind.Zachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/86Switch runtime dockerfile to use new Java 17 base image2023-11-29T21:17:02ZMartin Lowemartin.lowe@eclipse-foundation.orgSwitch runtime dockerfile to use new Java 17 base imagehttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/85Enhance the LoggingHelper to allow formatting of lists of strings2023-11-07T17:01:00ZZachary SabourinEnhance the LoggingHelper to allow formatting of lists of stringsZachary SabourinZachary Sabourinhttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/84Add working groups API retrieval to the efservices package2023-10-10T15:13:41ZMartin Lowemartin.lowe@eclipse-foundation.orgAdd working groups API retrieval to the efservices packageWith some updates made to project adopters, there are now more than 3 projects that pull in working groups from the working groups API. To reduce duplication, we should move the logic for fetching and caching the working groups data to t...With some updates made to project adopters, there are now more than 3 projects that pull in working groups from the working groups API. To reduce duplication, we should move the logic for fetching and caching the working groups data to this repo to reduce duplications.Martin Lowemartin.lowe@eclipse-foundation.orgMartin Lowemartin.lowe@eclipse-foundation.orghttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/83Move logging and cache resources to be non-application endpoints2024-03-20T14:12:26ZMartin Lowemartin.lowe@eclipse-foundation.orgMove logging and cache resources to be non-application endpointsWith our current setup, applications that don't have a proper root will never be able to properly use the caching and logging endpoints. To fix this, we should create custom non-application endpoints that exist outside of the normal scop...With our current setup, applications that don't have a proper root will never be able to properly use the caching and logging endpoints. To fix this, we should create custom non-application endpoints that exist outside of the normal scope of endpoints and can be mounted on a different root to make it available at runtime.
To do this, we would need to convert the core package to have a deployment/runtime split to allow for bindings of build time augments to include the above resources in the management interface. Details on how this is implemented is available in https://github.com/quarkusio/quarkus/blob/main/docs/src/main/asciidoc/management-interface-reference.adochttps://gitlab.eclipse.org/eclipsefdn/it/api/eclipsefdn-api-common/-/issues/82Investigate handling HTTP auth for EF staging services2024-03-20T14:10:52ZMartin Lowemartin.lowe@eclipse-foundation.orgInvestigate handling HTTP auth for EF staging servicesWhile testing projects-staging to validate behaviour, I noticed we still aren't handling HTTP auth for staging servers. This breaks our connections any time we attempt to connect to staging API services, which isn't ideal for proper inte...While testing projects-staging to validate behaviour, I noticed we still aren't handling HTTP auth for staging servers. This breaks our connections any time we attempt to connect to staging API services, which isn't ideal for proper integration testing. We should add an option to include an HTTP auth header if we set a config with a token.