Text Magic
We are aware of an issue affecting the delivery of SMS messages sent from OASIS. This is caused by a service disruption at our SMS provider, TextMagic, resulting from an outage at one of their external network providers. You can follow their progress on the TextMagic status page.
Users are affected in two ways:
Signing in with SMS multi-factor authentication — verification codes sent by text message may be delayed or may not arrive. If it is critical that you have access to OASIS during SMS outages such as this, we would recommend using a One Time Code (OTC) authenticator app rather than SMS.
Outbound SMS notifications — any text messages sent from OASIS (for example, appointment reminders or notifications to service users) may be delayed or fail to deliver while the issue is ongoing. Please consider using alternative contact methods for anything time-sensitive.
No OASIS systems or data are affected, this is limited to SMS delivery. TextMagic are investigating as a high priority, and we will post an update as soon as the issue is resolved.
OASIS Product Team
Following yesterday evening's issue affecting the delivery of SMS messages sent from OASIS, we can confirm that service has been restored. Our SMS provider, TextMagic, resolved the outage at their external network provider some time ago, and SMS delivery has been operating normally since.
SMS multi-factor authentication — verification codes are being delivered as normal, and users can sign in using SMS MFA without issue.
Outbound SMS notifications — text messages sent from OASIS are being delivered as normal. Please note that messages sent during yesterday evening's outage may not have been delivered, so if you sent anything time-sensitive during that period, we recommend checking with the recipient or re-sending where appropriate.
As a reminder, if reliable access to OASIS is critical for you, we recommend using a One Time Code (OTC) authenticator app as your MFA method, as this does not depend on SMS delivery.
Thank you for your patience, and apologies for the delay in posting this update.
OASIS Product Team
We are currently experiencing issues affecting SMS delivery and messaging services due to an ongoing incident with our upstream provider, TextMagic.
Customers may experience delayed or failed SMS notifications during this time. Our team is actively monitoring the situation and working with TextMagic to restore normal service as quickly as possible.
You can follow the upstream incident updates here: https://status.textmagic.com/incidents/qj2vk8bscmz5
We will continue to provide updates as more information becomes available.
We apologise for the inconvenience and appreciate your patience.
OASIS Product Team
We are aware users are experiencing increased error messages with OASIS from our internal monitoring system.
The team identified routine internal maintenance process not operating as expected.
OASIS Product Team
All systems now operational.
The root cause was a scaling operation which triggered an SQL instance restart, an operation which often happens during Microsoft's maintenance window without service interruption.
An ongoing process triggered a protection mechanism called rapid fail protection disabled the applications as a result of excessive error reporting.
We apologise for any inconvenience.
OASIS Product Team
We are aware users are experiencing access issues with OASIS and reports we have received from clients directly.
This will be observable when interacting with OASIS by saving and navigating around.
We are investigating the issue which we currently believe to be related to a Cloudflare outage.
We will provide an update soon.
OASIS Product Team
We identified the issue causing the access issue with OASIS, it relates to a Cloudflare outage. They haven't formally acknowledged this on their status portal but they are having ongoing issues.
OASIS is sporadically accessible we will continue to monitor any update from Cloudflare.
Thank you for you patience.
OASIS Product Team
Cloudflare have provided an update on their status page acknowledging a global network issues. Cloudflare Status
We will continue to monitor the progress and provide updates as necessary.
OASIS Product Team
OASIS is continuing to experience sporadic access issues as a result of the Cloudflare outage.
There have been an number of updates on the Cloudflare status page indicating they are seeing services recover as they continue remediation efforts. Cloudflare Status - Cloudflare Global Network experiencing issues
We will provide further updates as soon as we have them.
OASIS Product Team
Cloudflare have identified the issue impacting on their global network outage and are currently implementing a fix. Cloudflare Status - Cloudflare Global Network experiencing issues
We will provide further updates as they are available to us.
OASIS Product Team
We have received confirmation from Cloudflare that their global network issue has been resolved. OASIS services are now fully available and users are able to access the platform again.
This incident will remain open in a monitoring phase so that we can keep you informed of any changes to the current status as it remains open and being monitored by Cloudflare.
Cloudflare is a fundamental part of how the modern internet operates. Their infrastructure supports a very large proportion of global internet traffic and underpins the delivery of countless online services. As a result, issues within their network, although rare, can have far reaching effects that extend well beyond any single provider. While we allowed Cloudflare the necessary time to restore their systems, we were also preparing internally and assessing our options should the disruption have continued for a longer period.
We appreciate your patience throughout this incident.
OASIS Product Team
Following monitoring we are marking this issue as resolved as Cloudflare have closed of the issue.
OASIS Product Team
We have been made aware the OASIS report wizard is not applying filters correctly. The issue has been identified and will apply a patch to OASIS this evening.
We apologise for any inconvenience this has caused and will provide an update when a fix for this is deployed tonight.
OASIS Product Team
We have deployed the OASIS Report Wizard fixes to OASIS so filters should be apply correctly now.
OASIS Product Team
We will be performing planned maintenance on OASIS to install the renewed SSL certificate for the following domains:
- app.oasiscloud.co.uk
- secure.oasiscloud.co.uk
- analytics.oasiscloud.co.uk
- docs.oasiscloud.co.uk
Affected Services
OASIS Application
OASIS Document Services
OASIS Analytics
Reason for Maintenance
Renew the application SSL certificate.
Additional Information
During this maintenance window, we are not expecting any downtime. However, during the certificate swap users may be presented with certificate warning until the certificates have propagated.
Planned maintenance has completed
We will be performing planned maintenance on the Azure infrastructure to migrated some services which are being deprecated on the 30th of September by Microsoft which relate to Basic IP Address upgrades to Standard SKU.
During this maintenance window, we are not anticipating any downtime. The work is being carried out by Rackspace and IT Works will be available to verify upgrade completion and rollback decision making should the IP address upgrade be unsuccessful.
Affected Services:
-
VPN Gateway services between UK West and UK South data centres
Impact Level:
We are not anticipating any downtime as the upgrade does not impact on public facing services of OASIS.
Reason for Maintenance:
Microsoft are deprecating Basic IP address in Azure to Standard IP Addresses Upgrade Basic Public IP Address to Standard SKU in Azure | Microsoft Learn.
Additional Information:
The maintenance window is 1 hour with the potential to extend a further 2 hours if required.
OASIS Product Team
Planned maintenance has been completed.
OASIS Product Team
We are aware of an issue in OASIS when an organisation is using the functionality to stream documents for compatible files types is enabled. This issue will present as an error when you click on the uploaded document.
Our team are currently looking into this and will provide an update in the next hour.
Thank you for your patience.
OASIS Product Team
This issue has been resolved.
OASIS Product Team
We will be conducting some unplanned maintenance at 00:45 to deploy updates to some components used by OASIS to patch some issues we are observing.
These updates will cause a temporary disruption to page requests until the user navigates to another page.
We apologise for the short notice.
OASIS Product Team
We have start the update preparation work. We we expect to roll out the update in the next 10 minutes or so.
Thank you for your patience.
OASIS Product Team
The update process has been completed. Due to the nature of the updates we will be monitoring the platform throughout Friday and will apply any patches as required.
OASIS Product Team
We are aware users are experiencing performance issues with OASIS from our internal monitoring system and reports we have received from clients directly.
This will be observable when interacting with OASIS by saving and navigating around or an increased number of application errors presenting to users.
The team are currently working to identify the the cause of the performance issue and will provide an update to the incident soon.
OASIS Product Team
We identified the issue causing the performance issues with OASIS today. It relates to the weekly full back up processes in place for the databases. This is creating high data read load which is impacting the speed in which the OASIS application is able to communicate with the database. This has been exacerbated by additional patching required this morning which involved servers coming offline for updates and it takes time for the application to spool up.
Overall the performance is improving as the backup processes have moved onto smaller databases. We expect normal operations from around about 14:00. However, we will provide another update later this afternoon.
Thank you for you patience, we apologies for any interruption this may have caused.
OASIS Product Team
This issue has been marked as resolved as service performance returned to normal after further monitoring.
Thank you again for you patience, we apologies for any interruption this may have caused.
OASIS Product Team