We are aware of an issue affecting the delivery of SMS messages sent from OASIS. This is caused by a service disruption at our SMS provider, TextMagic, resulting from an outage at one of their external network providers. You can follow their progress on the TextMagic status page.
Users are affected in two ways:
Signing in with SMS multi-factor authentication — verification codes sent by text message may be delayed or may not arrive. If it is critical that you have access to OASIS during SMS outages such as this, we would recommend using a One Time Code (OTC) authenticator app rather than SMS.
Outbound SMS notifications — any text messages sent from OASIS (for example, appointment reminders or notifications to service users) may be delayed or fail to deliver while the issue is ongoing. Please consider using alternative contact methods for anything time-sensitive.
No OASIS systems or data are affected, this is limited to SMS delivery. TextMagic are investigating as a high priority, and we will post an update as soon as the issue is resolved.
OASIS Product Team
Following yesterday evening's issue affecting the delivery of SMS messages sent from OASIS, we can confirm that service has been restored. Our SMS provider, TextMagic, resolved the outage at their external network provider some time ago, and SMS delivery has been operating normally since.
SMS multi-factor authentication — verification codes are being delivered as normal, and users can sign in using SMS MFA without issue.
Outbound SMS notifications — text messages sent from OASIS are being delivered as normal. Please note that messages sent during yesterday evening's outage may not have been delivered, so if you sent anything time-sensitive during that period, we recommend checking with the recipient or re-sending where appropriate.
As a reminder, if reliable access to OASIS is critical for you, we recommend using a One Time Code (OTC) authenticator app as your MFA method, as this does not depend on SMS delivery.
Thank you for your patience, and apologies for the delay in posting this update.
OASIS Product Team
We are aware users are experiencing increased error messages with OASIS from our internal monitoring system.
The team identified routine internal maintenance process not operating as expected.
OASIS Product Team
All systems now operational.
The root cause was a scaling operation which triggered an SQL instance restart, an operation which often happens during Microsoft's maintenance window without service interruption.
An ongoing process triggered a protection mechanism called rapid fail protection disabled the applications as a result of excessive error reporting.
We apologise for any inconvenience.
OASIS Product Team
We are aware users are experiencing access issues with OASIS and reports we have received from clients directly.
This will be observable when interacting with OASIS by saving and navigating around.
We are investigating the issue which we currently believe to be related to a Cloudflare outage.
We will provide an update soon.
OASIS Product Team
We identified the issue causing the access issue with OASIS, it relates to a Cloudflare outage. They haven't formally acknowledged this on their status portal but they are having ongoing issues.
OASIS is sporadically accessible we will continue to monitor any update from Cloudflare.
Thank you for you patience.
OASIS Product Team
Cloudflare have provided an update on their status page acknowledging a global network issues. Cloudflare Status
We will continue to monitor the progress and provide updates as necessary.
OASIS Product Team
OASIS is continuing to experience sporadic access issues as a result of the Cloudflare outage.
There have been an number of updates on the Cloudflare status page indicating they are seeing services recover as they continue remediation efforts. Cloudflare Status - Cloudflare Global Network experiencing issues
We will provide further updates as soon as we have them.
OASIS Product Team
Cloudflare have identified the issue impacting on their global network outage and are currently implementing a fix. Cloudflare Status - Cloudflare Global Network experiencing issues
We will provide further updates as they are available to us.
OASIS Product Team
We have received confirmation from Cloudflare that their global network issue has been resolved. OASIS services are now fully available and users are able to access the platform again.
This incident will remain open in a monitoring phase so that we can keep you informed of any changes to the current status as it remains open and being monitored by Cloudflare.
Cloudflare is a fundamental part of how the modern internet operates. Their infrastructure supports a very large proportion of global internet traffic and underpins the delivery of countless online services. As a result, issues within their network, although rare, can have far reaching effects that extend well beyond any single provider. While we allowed Cloudflare the necessary time to restore their systems, we were also preparing internally and assessing our options should the disruption have continued for a longer period.
We appreciate your patience throughout this incident.
OASIS Product Team
Following monitoring we are marking this issue as resolved as Cloudflare have closed of the issue.
OASIS Product Team
We are aware users are experiencing performance issues with OASIS from our internal monitoring system and reports we have received from clients directly.
This will be observable when interacting with OASIS by saving and navigating around or an increased number of application errors presenting to users.
The team are currently working to identify the the cause of the performance issue and will provide an update to the incident soon.
OASIS Product Team
We are still investigating the root cause of the performance issue. However, OASIS is operating as expected. We will post an update on the root cause soon.
Thank you for you patience.
OASIS Product Team
We are pleased to confirm that this issue has now been resolved. The investigation found that the cause was increased CPU load from encryption processes used when removing data from completed client migrations.
To help prevent this from happening again, these processes will now be carried out outside periods of peak use on OASIS.
Thank you for your patience and understanding.
OASIS Support Team
We have been made aware the OASIS report wizard is not applying filters correctly. The issue has been identified and will apply a patch to OASIS this evening.
We apologise for any inconvenience this has caused and will provide an update when a fix for this is deployed tonight.
OASIS Product Team
We have deployed the OASIS Report Wizard fixes to OASIS so filters should be apply correctly now.
OASIS Product Team
We are aware users are experiencing performance issues with OASIS from our internal monitoring system and reports we have received from clients directly.
This will be observable when interacting with OASIS by saving and navigating around or an increased number of application errors presenting to users.
The team are currently working to identify the the cause of the performance issue and will provide an update to the incident soon.
OASIS Product Team
We have identified the server causing the system timeouts. We have brought this system offline while we restart the services and will attempt to integrate back into the platform as soon as the processes are online.
Thank you for you patience.
OASIS Product Team
The server responsible for the issue has been reinstated. We will continue to monitor OASIS closely for any unusual behaviour.
We apologise for any inconvenience this incident may have caused.
OASIS Product Team
This issue has now been resolved.
We are aware of an issue with users using linked accounts being asked to login when moving between deployments.
The team are investigating this issue and will provide an update shortly.
Apologies for any inconvenience caused.
OASIS product Team
We have rolled out an update to address the issue with linked account operations.
The team will monitor the issue for the next 24 hours.
OASIS Product Team
Issue has been resolved.
We are aware of an issue in OASIS when an organisation is using the functionality to stream documents for compatible files types is enabled. This issue will present as an error when you click on the uploaded document.
Our team are currently looking into this and will provide an update in the next hour.
Thank you for your patience.
OASIS Product Team
This issue has been resolved.
OASIS Product Team
We are aware users are experiencing performance issues with OASIS from our internal monitoring system and reports we have received from clients directly.
This will be observable when interacting with OASIS by saving and navigating around or an increased number of application errors presenting to users.
The team are currently working to identify the the cause of the performance issue and will provide an update to the incident soon.
OASIS Product Team
We identified the issue causing the performance issues with OASIS today. It relates to the weekly full back up processes in place for the databases. This is creating high data read load which is impacting the speed in which the OASIS application is able to communicate with the database. This has been exacerbated by additional patching required this morning which involved servers coming offline for updates and it takes time for the application to spool up.
Overall the performance is improving as the backup processes have moved onto smaller databases. We expect normal operations from around about 14:00. However, we will provide another update later this afternoon.
Thank you for you patience, we apologies for any interruption this may have caused.
OASIS Product Team
This issue has been marked as resolved as service performance returned to normal after further monitoring.
Thank you again for you patience, we apologies for any interruption this may have caused.
OASIS Product Team
We are experiencing some issues with calendars in OASIS this morning following an update to some application dependencies. The team are investigating the root cause of the issue and will provide an update later today regarding the plans to remediate this issues being experienced.
OASIS Product Team
We have identified the specific dependency causing the issue. This will be deployed in the next few hours to the live system.
An update will be provided on this incident when this fix has been deployed.
OASIS Product Team
We have deployed update the relevant dependencies which will resolve the access issues to the calendars in OASIS.
Apologies for any inconvenience caused.
OASIS Product Team