...
For an effective Open Banking ecosystem to thrive, there is a need for industry-wide benchmarks, referred to as the "Recommended Benchmarks". Since these guidelines are based on global best practices, these will be reviewed on a regular basis in consultation with industry stakeholders. ASPSPs should aim to adhere refer to the Recommended Benchmarks recommended best practices for their interfaces as a best practice.
Additionally, the guiding principle of these benchmarks is that ASPSPs must ensure (at least) parity between the availability and performance of the best performing interface (that they own and maintain to interact directly with customers) and that of their open banking interface which may be leveraged by AISP/PISPs.
...
The definition of a period of availability is a period of time when any of the API end points defined in the Bahrain Open Banking Framework (BOBFBahrain OBF) are able to reliably provide a successful response to an appropriately constructed request.
...
This section sets out a minimum of two KPIs for availability that an ASPSP should have in place for each API. The following table explains these KPIs in detail and provides guidance on how they should be calculated. (The downtime mentioned is set for initial implementation of Bahrain OBF and may be revised on periodic basis)
Table 1 – Key Indicators based on best practices for Availability
Title | Requirement | Calculation Guidelines | Recommended Benchmark |
The uptime per day of all interfaces | For the purpose of calculating the availability indicators, the ASPSP should: a) calculate the percentage uptime as 100% minus the percentage downtime | For each 24 hour period, uptime is calculated as 100% minus the total percentage downtime in that period. | A quarterly uptime of 99.5%[1] |
The downtime per day of all interfaces | For the purpose of calculating the availability indicators, the ASPSP should: a) calculate the percentage downtime using the total number of seconds the API was down in a 24 hour period, starting and ending at midnight
b) count the interface as ‘down’ when five consecutive requests for access to information for the provision of payment initiation services, account information services are not replied to within a total timeframe of 30 seconds, irrespective of whether these requests originate from one or multiple PISPs/AISPs. In such a case, the ASPSP should calculate downtime from the moment it has received the first request in the series of five consecutive requests that were not replied to within 30 seconds, provided that there is no successful request in between those five requests to which a reply has been provided | Downtime should be calculated as follows:
At a minimum, downtime should be measured if:
This should include any vendor/supplier failures in the case where the ASPSP has contracted the vendor/supplier to deliver a related service However, this should exclude errors resulting from issues outside of the ASPSP's direct control, such as issues with AISP/PISP software, infrastructure or connectivity. | A quarterly downtime of 0.5%. (approx. 11 hours per quarter, or just under four hours per month, to allow for planned releases, updates, and also any unplanned downtime) |
[1] This is the benchmark for Business as usual/normal conditions and not in cases of force majeure events such as natural calamities, war, cyber threats etc.
2.1.2 Performance
The BOBF Bahrain OBF defines a number of endpoints which should be made available by ASPSPs. The Performance of API end points should be measured in response time of individual API requests from receipt of request to delivery of response. While all supported endpoints should be included by ASPSPs when calculating error rates, for reporting response times the consent endpoints should be ignored, as these are not considered part of the process of 'providing the information requested' to the AISP/ PISP for account information or payment initiation.
This section sets out a minimum of four KPIs for performance that an ASPSP should have in place for each of its API. The following table explains these KPIs in detail and provides guidance on how they should be calculated.
Table 2 – Key Indicators based on best practices for Availability
Title | Requirement | Calculation Guidelines | Recommended Benchmark |
PISP response time | For the purpose of calculating the performance indicators, the ASPSP should: a) calculate the daily average time (in milliseconds) taken, per request, for the ASPSP to PISP with all the information requested | The "time taken per request" should be calculated for each day using the mean value of Time to Last Byte (TTLB) measured in milliseconds, starting from the time that each endpoint request has been fully received by the ASPSP and stopping when the last byte of the response message has been transmitted to the PISP. The following API endpoints should be included when calculating PISP response times, for each endpoint supported by the ASPSP:
The ASPSPs signed response to the POST will inherently act as proof of receipt of the payment order by the ASPSP, which will enable the PISP to log a reference and the date of this receipt. Both the POST and the GET endpoints contain all information relating to the payment, which, depending on the payment type, should include reference, amount, exchange rate, charges, and status (which may change between POST and any subsequent GET). The POST endpoints above cater for the ASPSP to make the information available to the PISP immediately after receipt of the payment order, and the provision of all information on the initiation of the payment transaction and all information accessible to the ASPSP regarding the execution of the payment transaction. The GET endpoints cater for the ASPSP to provide confirmation to the PISP that payment initiation has been successful, in order to enable the PISP to provide this information to the user/customer. Since different endpoints will have different payload sizes for request and response (especially relevant for bulk/batch payment endpoints involving large files), and in order to facilitate a 'like for like' comparison with user/customer interfaces, ASPSPs should report on the average time per megabyte (MB). This can be calculated by dividing the total response time in milliseconds by the total payload response size in MB, across all API calls for all API endpoints for each day. | An average TTLB of 750 milliseconds per response for all but bulk/batch payments |
AISP response time | For the purpose of calculating the performance indicators, the ASPSP should: a) calculate the daily average time (in milliseconds) taken, per request, for the ASPSP to provide the AISP with all the information requested | The "time taken per request" should be calculated for each day using the mean value of Time to Last Byte (TTLB) measured in milliseconds, starting from the time that each endpoint request has been fully received by the ASPSP and stopping when the last byte of the response message has been transmitted to the AISP. The following API endpoints should be included when calculating AISP response time, for each endpoint supported by the ASPSP:
Since different endpoints will have different payload sizes for request and response, and in order to facilitate a 'like for like' comparison with user/customer interfaces, the CBB recommends that ASPSPs also report on the average time per megabyte (MB). This can be calculated by dividing the total response time in milliseconds by the total payload response size in MB, across all API calls for all API endpoints for each day. | An Average TTLB of 750 milliseconds per response, or per page of results for up to 100 records for larger payloads. In practice, all but transactions and statements are likely to be small payloads without pagination |
Confirmation of Funds (CoF) response time (PISP) | For the purpose of calculating the performance indicators, the ASPSP should:
a) calculate the daily average time (in milliseconds) taken, per request, for the ASPSP to provide the PISP with a ‘yes/no’ confirmation
| The "time taken per request" should be calculated for each day using the mean value of Time to Last Byte (TTLB) measured in milliseconds, starting from the time that each endpoint request has been fully received by the ASPSP and stopping when the last byte of the response message (i.e. the 'yes/no' confirmation) has been transmitted to the PISP. The following API endpoints should be included when calculating CoF response times for PISP:
| An average TTLB of 300 and a max of 500 ms per response |
Daily error response rate | For the purpose of calculating the performance indicators, the ASPSP should: a) Calculate the daily error response rate – calculated as the number of error messages concerning errors attributable to the ASPSP sent by the ASPSP to the PISPs, or AISPs per day, divided by the number of requests received by the ASPSP from PISPs or AISPs in the same day
| It is not possible for ASPSPs to respond to AISPs/PISPs with an error message where no TLS (Transport layer security) session has been established. However, ASPSPs should still be able to respond, measure and report on errors relating to endpoint calls and all functional API calls. The error response rate should be calculated as the total number of all 5xx HTTP status codes from all API endpoints per day, divided by the total number of AISP/PISP API requests received across all of these endpoints in the same day, and expressed as a percentage. Errors based on 4xx HTTP status codes are largely attributable to AISP/PISP or user/customer actions or failures, and hence should not be included here. | An average of 0.5% across all endpoints |
...
# | Field Name | Description/ Definition |
1 | Performance and Availability | |
1.1 | ReportDate | The reported date for each calendar day. |
1.2 | Entity Name | Reporting entity’s name. |
1.3 | Endpoint ID | Reported EndPoint ID as defined in the API Endpoint List of the Reporting Template. ASPSPs/ AISPs/ PISPs must only report endpoints that have gone live in their systems. |
1.4 | Uptime | Uptime per each individual endpoint in hours and minutes. (Elapsed time) Calculated as 100% minus the total percentage downtime for each day. |
1.5 | Planned Downtime | Any planned duration that the API endpoints become unavailable. For the avoidance of doubt, this extends to include all systems that are required for the relevant endpoint to be fully functional. |
1.6 | Unplanned Downtime | Any unplanned duration that the API endpoints become unavailable due to technical faults or any other reasons. For the avoidance of doubt, this extends to include all systems that are required for the relevant endpoint to be fully functional. |
1.7 | Max Payment Initiations Per Second (PIPS) | This field is only applicable to PIS endpoints. For AIS endpoints it should be populated with NULL. The maximum number of successful payment initiations per second (PIPS) for the payment order POST API calls. The PIPS must be reported as whole numbers. |
1.8 | Average TTLB Response Time (Time to Last Byte) | The average (mean) value across all values of Time to Last Byte (TTLB) response time for each API endpoint. The response time clock should start at the point the endpoint call is fully received by the ASPSP and should stop at the point the last byte of the response message is transmitted to the AISP or PISP. |
1.9 | Average TTFB Response Time (Time to First Byte) | The average (mean) value across all values of Time to First Byte (TTFB) response time for each API endpoint. The response time clock should start at the point the endpoint call is fully received by the ASPSP. |
1.10 | Total Number of API calls | This is the total number of API calls per day per endpoint. |
1.11 | Total TTLB Response Time (Time to Last Byte) | This is the sum of all the TTLB responses of all endpoint calls of each endpoint type. For the avoidance of doubt, this is the sum of all the TTLB response times generated by the Total Number of API calls for each endpoint. |
1.12 | Total TTFB Response Time (Time to First Byte) | This is the sum of all the TTFB responses of all endpoint calls of each endpoint type. For the avoidance of doubt, this is the sum of all the TTFB response times generated by the Total Number of API calls for each endpoint. |
1.13 | Total ResponsePayload Size | This is the sum of the payload of all the response messages for all endpoint calls of each endpoint type. For the avoidance of doubt, this is the sum of all the payloads of the response messages generated by the Total Number of API calls for each endpoint. |
2 | Authentication | |
2.1 | ReportDate | The reported date for each calendar day. |
2.2 | Entity Name | Reporting entity’s name. |
2.3 | Authentication Type | This is the type of authentication journeys provided by the ASPSP. It will include 'redirection' and where implemented 'decoupled' model. |
2.4 | API Type | This is the type of services that are being reported for the efficacy of the authentication journey. It includes Account Information Services (AIS) and Payment Initiation Services (PIS). |
2.5 | API Request AISP / PISP Channel | This is the reported AISP / PISP channel for initiating the AIS or PIS consent. This may be provided by AISPs / PISPs in the endpoint Request Header under the field 'x-customer-user-agent'. If the string cannot be mapped to a browser, then it will probably be a mobile app. |
2.6 | ASPSP Authentication Channel | This is the reported ASPSP Authentication channel. It can be web-based (Web) or using the mobile banking app (App). |
2.7 | Consents requiring Authentication | The total number of user/customer consents to require authentication at the ASPSP for the particular combination of authentication type, API type, AISP / PISP channel and ASPSP authentication channel. |
2.8 | Authentications Attempted by users/customers | The total number of authentications that have been attempted by the users/customers (not abandoned). This means that the Users/ Customers have tried to authenticate according to the authentication method required providing biometrics, username, passwords etc. |
2.9 | Authentications Abandoned by users/customers | The total number of user/customer consents to require authentication that has been abandoned by the users/customers. |
2.10 | Authentications Succeeded | The total number of consents requiring authentication that have completed authentications by users/customers and the authentications have succeeded. |
2.11 | Authentications Failed | The total number of consents requiring authentication that have completed authentications by users/customers and the authentications have failed. |
2.12 | Confirmations Required | The total number of consents requiring authorization, that after successful authentication, required a confirmation step. |
2.13 | Confirmations Accepted by user/customer | The total number of successful authentications that required a confirmation step and have been accepted by users/customers. This means that the users/customers proceeded with the access request or the payment order submission and did not cancel the process. |
2.14 | Confirmations Rejected by users/customers | The total number of successful authentications that required a confirmation step and have been rejected by users/customers. This means that the users/customers cancelled the process and did not proceed with the access request or the payment order submission. |
3 | User/ Customer Volumes | |
3.1 | ReportMonth | Reported calendar month and year. |
3.2 | Entity Name | Reporting entity’s name. |
3.3 | Retail/Business user/customer | This identifies Retail and Business users/customers for separate reporting. |
3.4 | user/customer used AIS Services for the first time | The number of unique users/customers who have authorised access to their account(s) to one or more AISPs for account information services for the first time during the reporting period. For the avoidance of doubt, this refers to new consent/authorisations and not re-authorisations. |
3.5 | Total users/customers used AIS Services | The total number of unique users/customers who: a. have authorised access to their account(s) to one or more AISPs for account information services during the reporting period for the first time b. have previously authorised access to their account(s) to one or more AISPs for account information services and had their account accessed at least once by any of the AISPs during the reporting period c. have previously authorised access to their account(s) to one or more AISPs for account information services and had to re-authenticate to refresh the account access at least for one of the AISPs during the reporting period |
3.6 | user/customer used PIS Services for the first time | The number of unique user/customer who have authorised a payment initiation of any type from any of their account(s) via one or more PISPs for the first time during the reporting period. For the avoidance of doubt, a user/customer initiating a payment (e.g. single domestic) using a PISP A from a PCA A and then initiating another payment of different type (e.g. international) using a PISP B from a PCA B within the same ASPSP brand , should not be double-counted. For business customers, unique users/customers should refer to all employees of the business who have separate authentication credentials and can be identified separately. Multi-banked users/customers cannot be identified so they may be double-counted by different ASPSPs. |
3.7 | Total users/customers used PIS Services | The total number of unique users/customers who: a. have authorised a payment initiation of any type from any of their account(s) via one or more PISPs for the first time during the reporting period b. have authorised a payment initiation of any type from any of their account(s) via one or more PISPs during the reporting period and have initiated payments using PIS services before. |
3.8 | Unique users/customers used both AIS and PIS Services for the first time | Users/Customers accessing both AIS and PIS services using the same authentication credentials. |
3.9 | Total unique users/customers used both AIS and PIS Services | Total users/customers accessing both AIS and PIS services using the same authentication credentials. |
3.10 | Total new users/customers for Online Banking | The number of unique users/customers who have been granted access to Online Banking for the first time. |
3.11 | Total new users/customers for Mobile Banking | The number of unique users/customers who have been granted access to Mobile Banking for the first time. |
3.12 | Total number of users/customers used Online Banking | The total number of unique users/customers who have used the Online Banking service for either accessing account information or initiating a payment. |
3.13 | Total number of users/customers used Mobile Banking | The total number of unique users/customers who have used the Mobile Banking service for either accessing account information or initiating a payment. |
4 | AISP/PISP Volumes | |
4.1 | ReportMonth | Reported calendar month and year. |
4.2 | Entity Name | Reporting entity’s name. |
4.3 | Total AISPs Registered (at 1st of the month) | This is the cumulative total number of AISP’s, that have been already been on-boarded into the live environment (i.e. production environment) with the ASPSP as at the 1st of the reported month. |
4.4 | AISP Additions | This is the number of AISPs that have been on-boarded into the live environment (i.e. production environment) during the reported month. |
4.5 | AISP Deregistrations | This is the number of AISPs, that have been deregistered with the ASPSP during the reported month. |
4.6 | Cumulative Monthly number of AISPs | This is the cumulative total number of AISP’s, that have been already been on-boarded into the live environment (i.e. production environment) with the ASPSP as at the end of the reported month. |
4.7 | Total PISPs Registered (at 1st of the month) | This is the cumulative total number of PISP’s, that have been already been on-boarded into the live environment (i.e. production environment) with the ASPSP as at the 1st of the reported month. |
4.8 | PISP Additions | This is the number of PISPs that have been on-boarded into the live environment (i.e. production environment) during the reported month. |
4.9 | PISP Deregistrations | This is the number of PISPs, that have been deregistered with the ASPSP during the reported month. |
4.10 | Cumulative Monthly number of PISPs | This is the cumulative total number of PISP’s, that have been already been on-boarded into the live environment (i.e. production environment) with the ASPSP as at the end of the reported month. |
5 | Daily Volumes | |
5.1 | ReportDate | The reported date for each calendar day. |
5.2 | Entity Name | Reporting entity’s name. |
5.3 | Endpoint ID | Reported EndPoint ID as defined in the API Endpoint List of the Reporting Template. ASPSPs must only report endpoints that have gone live in their systems. |
5.4 | Successful API Calls (200, 201 or 204 codes) | This is the total number of successful endpoint calls for each endpoint that have been received successfully by the ASPSP brand and generated a HAISP / PISP Status Code of 200, 201 or 204 depending on the HTTP method of the endpoints. |
5.5 | Failed API Calls Business Reasons (4xx Codes) | This is the total number of failed endpoint calls for each endpoint that have been received by the ASPSP brand and failed due to business rules reasons generating an HTPP Status Code of 4xx). |
5.6 | Failed API Technical Calls Reasons (5xx Codes) | This is the total number of failed endpoint calls for each endpoint that have been received by the ASPSP brand and failed due to technical reasons (generating an HTPP Status Code of 500 Internal Server Error). |
5.7 | API Calls Rejected Status | This is the total number of rejected endpoint calls that have been rejected for each defined endpoint which might be due payment/account consent being rejected due to authorisation failing or consent authorisation being rejected, payment initiation being rejected as part of proceeding checks such as technical validation and customer profile. |
...
Where ASPSPs support more than one API version in production, each version must be reported separately. For example, v3.0 and v3.1 must be reported separately.
In the absence of any relevant information the ASPSPs must return with “NIL” response.
2.3.2 AISP / PISP Reporting
...
ASPSPs are required to provide a Testing Facility or sandbox to allow AISPs / PISPs to undertake connection and functional testing of their products and services using non-user/customer (i.e. “dummy”) data. The issues and problems which are identified within this testing process, as well as feedback and engagement from the AISP / PISP community, are useful for ASPSPs in alerting them to potential issues within testing that may also be encountered within the production environment. This can be used to identify and address issues early on. ASPSPs will be required to provide details and information on the outputs of their testing to their accreditation body as part of their application. This facility should[1] provide an
This facility should[2] provide an accurate reflection of the live environment, and give AISP / PISP developers access to the following:
...
Availability and performance: The facility is not expected to handle production volumes (i.e. is not expected to be used by ASPSPs or AISPs / PISPs for stress testing), however, it should have sufficient availability, capacity, performance and other characteristics to facilitate effective and realistic connection and functional AISP / PISP testingReadiness: The facility must enable AISPs / PISPs to start testing their technical solutions at least six months prior to the date of launch of the ASPSP’s dedicated interface
Ongoing access: The facility should remain as an ongoing facility and to support future development or changes to the dedicated interface at least 3 months prior to implementation of such changes
Support: The facility should have an appropriate level of support to enable communication of problems or issues by AISPs / PISPs to ASPSPs and to provide efficient and effective solutions
Documentation: ASPSPs must publish externally a summary of the specification of the testing facility on their website including access details and test coverage
...
ASPSPs can provide advance notice for future planned downtime and submit real time updates related to downtime (planned or unplanned) that currently impact AISPs / PISPs and the subsequent reinstatement of service. It is not expected that ASPSPs raise tickets for very short lived periods of unplanned downtime (e.g. when full service is likely to be restored before the ticket has been raised), although all downtime should be reported as per section 2 above.
Based on global best practices, planned downtime should be given with at least five business days’ prior to the event. Apart from cancelling the planned downtime, no changes should be made to the planned downtime notification within the five business day period. Where practical, ASPSPs should give advance notice via their own website, developer portal and to the CBB of any planned downtime one calendar month in advance.
...
The Bahrain Open Banking Framework (BOBFBahrain OBF) will continue to evolve over time to cater for potential improvements/clarifications, agreed Open Banking roadmap requirements and approved changes (which may include adding new functionality, fixing defects, and errata). Where possible, the CBB will schedule new versions of the BOBF Bahrain OBF with sufficient time for change so that ASPSPs can plan ahead and build new APIs to this plan, this will therefore reduce development and support costs for all participants and increase adoption.
...
ASPSPs must be compliant with the latest version of Bahrain’s Open Banking Framework as published by the CBB in order to meet both regulatory and commercial requirements
ASPSPs must publish technical specification documentation for their dedicated interface, well in advance before the target date of the market launch of the dedicated interface.
ASPSPs must give AISPs / PISPs at least three months’ notice of any change to the technical specifications, unless the changes can be rolled out sooner, in which case ASPSs ASPSPs must notify AISPs/PISPs on the change as soon as possible . In practice, this means that ASPSPs must give such notice if they are planning to introduce any updates to any component of the dedicated interfaces, Any change may be implemented in an emergency situation (e.g. in the case where there is a security issue) without such notice, and in such situations ASPSPs must document emergency situations where changes are implemented and make the documentation available to competent authorities on request
ASPSPs must also make available a testing facility (sandbox) well in advance before the target date of the market launch of the dedicated interface. ASPSPs should ensure that any changes are made available in the testing facility as soon as possible to allow AISPs / PISPs to test against the updated technical specifications. In practice, this means that ASPSPs should consider the impact of proposed changes on their testing facility in order to ensure that the testing facility enables the same functionality as the dedicated interface, in the context of such changes. As such, ASPSPs should endeavor to make any changes to the testing facility available to AISPs / PISPs at least three months before changes are implemented to ensure AISPs / PISPs, can continue to effectively test
ASPSPs should maintain multiple live/active versions of each interface (e.g. one for each supported release)
Where an ASPSP decides to implement a new version of any component of the BOBF Bahrain OBF they should implement each new major version within six months, and each new minor version within three months of the BOBF Bahrain OBF being published by the CBB
Together with the requirements for ASPSPs to notify AISPs / PISPs of any changes any AISP / PISP will, except in an emergency, always have at least three months’ notice before being required to update their systems
...