/
Availability, Performance and Usage Benchmarks

Availability, Performance and Usage Benchmarks

1. Introduction

This section provides clarity on expectations of participants from an availability and performance perspective, defining benchmarks for various requirements. The OFP will provide all reports via a portal, and therefore LFIs are not expected to provide reports. This section provides clarity on how the reported benchmarks will be measured.

In all instances of an interaction referenced between an LFI and a TPP, it is implied that this interaction is facilitated via the OFP.

1.1 User Needs

  • TPPs need to rely on highly available and well performing APIs provided by LFIs via the OFP so that they can offer reliable services to Users.

  • CBUAE will monitor the availability and performance of LFIs' APIs via the OFP to oversee and ensure the ecosystem's health and take any corrective actions if appropriate.

  • CBUAE will monitor the usage of Open Finance services to assess the adoption by Users.

1.2 Purpose

The purpose of this section is to set out the following:

  • Key Performance Indicators (KPIs) and benchmarks for LFI's API Availability 

  • Key Performance Indicators (KPIs) and benchmarks for LFI's API Performance

1.3 Benefits

Based on the above, CBUAE can apply product management methodologies and make informed decisions to improve the development and adoption of Open Finance. This may include the following:

  • Collecting multiple sources of data to ensure validation and consistency of all reported data;

  • Sharing information and insights with the ecosystem;

  • Use of forecasting methods to predict growth or users and API call volumes to perform capacity planning and ensure sufficient support services are in place;

  • Informing further enhancements or amendments to the Standard to (better) enable existing or new functionality; and

  • Thereby acting as a reference for the evaluation of the impact of Open Finance in the UAE.

2. API Availability

The following table explains the KPIs used to measure the availability of an LFI's APIs and sets the required benchmark.

Title

Requirement

Calculation

Benchmark

Availability of the API

The percentage uptime of the API, calculated as 100% minus the percentage downtime for each day

For every 24 hours (from 00:00:00 to 23:59:59 every day), we calculate 100% minus the total percentage downtime in that period.

Downtime will be calculated as follows:

  • The total number of concurrent seconds per API calls per 24-hour period, starting and ending at midnight, that any element of the API interface is not available divided by 86,400 (the number of seconds in 24 hours) and expressed as a percentage.

  • The clock for downtime will start immediately after the first 'failed' API request or response has been logged and will stop once a 'successful' API response has been logged. In addition, any unexpected connectivity issues causing the LFI not to be able to receive any API calls from the OFP will also be measured as downtime.

Downtime will be measured if:

  • The OFP authorization and/or resource server is not fully accessible and accepting all valid TPP requests.

  • Any LFI downstream system required to support these API endpoints is also not responding in a way that affects the availability of the OFP authorization and/or resource servers.

  • Any of the LFI screens and/or functionality of the customer authentication flow is unavailable to enable customers to grant TPPs access to their account(s).

  • This will include all 5xx errors.

  • Errors based on 4xx HTTP status codes are mainly attributable to TPP or User actions or failures and hence will not be included in the calculation.

  • This will include both planned and unplanned downtime during each day.

  • Even if this only affects some TPPs and/or customers, downtime will still be reported, i.e., partial downtime will still be measured as downtime.

  • This will be attributed to the LFI for any vendor/supplier failures if the LFI has contracted the vendor/supplier to deliver a related service, e.g., the LFI's hosting provider.

However, this will exclude errors resulting from issues outside the LFI and OFP’s direct control, such as issues with TPP software, infrastructure, or connectivity.

Uptime of 99.5%.

This allows for just under four hours per month of downtime to cover planned releases, updates, and unplanned downtime.

3. Error Rates

Title

Requirement

Calculation

Error rate

Daily error rate

For every 24 hours (from 00:00:00 to 23:59:59 every day), the daily error rate will be calculated as the number of error messages concerning errors attributable to the LFI sent by the LFI to the OFP per day, divided by the number of requests received by the LFI from the OFP in the same day.

It is not possible for LFIs to respond to the OFP with an error message where no TLS session has been established. However, LFIs SHOULD still be able to respond to errors relating to all FAPI endpoint calls and all functional API calls relating to the Standard.

The error rate will be calculated as the total number of all 5xx HTTP status codes from all API endpoints per day, divided by the total number of OFP API requests received across all endpoints on the same day, and expressed as a percentage.

Errors based on 4xx HTTP status codes are mainly attributable to TPP or User actions or failures and hence will not be included in the calculation.

4. API Performance

The following table explains the KPIs used to measure the performance of an LFI's APIs and sets the required benchmark.

Title

Requirement

Calculation

Benchmark

API performance

Daily average time (in milliseconds) taken, per request, for the LFI to provide the TPP via the OFP with all the requested information for the following API endpoints:

Bank Data

GET /account-access-consents

GET /accounts

GET /accounts/{AccountID}/balances

GET /accounts/{AccountID}/beneficiaries

GET /accounts/{AccountID}/consents

GET /accounts/{AccountID}/direct-debits

GET /accounts/{AccountID}/parties

GET /accounts/{AccountID}/product

GET /accounts/{AccountID}/scheduled-payments

GET /accounts/{AccountID}/standing-orders

GET /accounts/{AccountID}/transactions

Bank Service Initiation

GET /payment-consents

GET /payment-consents{ConsentId}

GET /payment-consents{ConsentId}/refund

POST /payment-consents/{ConsentId}/file

PATCH /payment-consents{ConsentId}

GET /payments/{PaymentId}

POST /payments

HEAD /payments

GET /file-payments/{PaymentId}

POST /file-payments

HEAD /file-payments

GET /file-payments/{PaymentId}/report

Confirmation of Payee

POST /confirmation

POST /discovery

Insurance

GET /insurance-consents

GET /insurance-consents{ConsentId}

PATCH /insurance-consents{ConsentId}

GET /insurance-policies

GET /insurance-policies/{InsurancePolicyId}

 

 

For every 24 hours (from 00:00:00 to 23:59:59 every day), the average API response time will be calculated in milliseconds (ms) for each API endpoint as follows:

Avg = T / V

Where:

Avg = average API response time in ms

T = total API response time in ms for all API calls for each API endpoint in a given day

V = total volume of API calls for each API endpoint in a given day

The benchmark will be apportioned in an equal split between the OFP and the LFI, subject to amendment in a future version of the Standard.

API response time is measured as the Time to Last Byte (TTLB) for each API endpoint, defined as the period in ms from when the LFI receives the API endpoint request from the OFP until the requesting TPP gets all the data in the response message via the OFP.   

The response time will not include the time the User takes to complete any authentication process.

Service Initiation

An average TTLB of 500ms per response across all service initiation APIs, excluding batch/file payments.

Bank Data

An average TTLB of 500ms per response across all bank data APIs, or per page of up to 100 records for large datasets.

5. Payment Execution

Title

Requirement

Calculation

Benchmark

Payment execution

As per Aani scheme rules

 

3 seconds per payment