Dashboards stale - nightly refresh failed
Ticketclosed
Veenendrecht Payments · Dashboards stale - nightly refresh failed
Nightly refresh failed due to rate limiting on Xero and Salesforce queries. Kate Ormerod identified the cause and implemented exponential backoff retry logic; data became current by 23 Jul 2026.
What happened
What this says
As at Fri 9 Oct 2026Nightly refresh failed due to rate limiting on Xero and Salesforce queries. Kate Ormerod identified the cause and implemented exponential backoff retry logic; data became current by 23 Jul 2026.
Problem: The nightly refresh did not complete on 21 Jul 2026, leaving dashboards with stale data. Merchant success and merchant risk teams needed current data for business decisions. Handling: Kate escalated to the platform team. Root cause was identified as rate limiting when querying Xero and Salesforce. Exponential backoff retry logic was implemented and the refresh re-ran successfully. Watch: Resolved with architectural improvement. Customer asked for confirmation that recurrence is prevented. Monitor retry logic effectiveness in handling similar scenarios.
Details
Updated 9 Oct 26Trail
Public reply · Eline Bakker 21 Jul 2026 12:21The dashboards haven't refreshed. Still showing data from yesterday. Are you seeing anything on your end? Our teams need current data to move forward with merchant decisions. Thanks- created · Eline Bakker 21 Jul 2026 13:42The nightly refresh hasn't completed since last night. Our dashboards are still showing yesterday's data. This is quite urgent for us because our merchant success and merchant risk teams rely on current data to make decisions about which merchants to work with and any disputes or issues. We need this fixed as soon as possible. What's the status?
- Public reply · Kate Ormerod 21 Jul 2026 14:09Hi Eline, thanks for raising this. I can confirm the nightly refresh did not complete successfully. I'm escalating this to our platform team right now so they can investigate and fix it. I'll keep you updated every step of the way. Thanks for your patience, Kate
- Internal note · Kate Ormerod 23 Jul 2026 09:26Root cause identified: refresh process timed out when querying Xero and Salesforce due to rate limiting. Implemented exponential backoff retry logic in refresh pipeline. Refresh re-ran successfully 06:45 yesterday. All data now current as of the nightly refresh.
- Public reply · Kate Ormerod 23 Jul 2026 09:34Hi Eline, we've found and fixed the issue. The nightly refresh was hitting rate limits when querying a couple of your source systems. We've implemented a retry mechanism that handles these situations gracefully. The refresh has now run successfully and your dashboards have the latest data. You're all set to move forward. Thanks, Kate
- Public reply · Eline Bakker 23 Jul 2026 16:50Perfect, thanks Kate. Dashboards are showing current data now. Just to confirm, this won't happen again if our source systems temporarily block requests? And we can proceed with confidence? Groetjes, Eline
- Status change · 27 Jul 2026 23:59