Service update – 6:30pm – Platform fully accessible again
We’re ever so sorry to have caused additional stress today at what we know is already a very difficult time. Demand continued to be off the charts (about 5,000 requests a second) in the morning “rush hour”. We had hoped that the changes we made last night, increasing our infrastructure to over 16x our usual capacity, would do the trick, but encountered separate problems this morning with a different part of the system that only appeared as a result of the increased scale elsewhere (Redis, for the techies among you, which we use to share sessions across our web server pool). Once we had that back online this afternoon – re-architected and hugely scaled up – we saw knock-on effects of the overprovisioning causing some of our web servers to hit limits on how many things they could connect to at once. This caused about 5% of requests to fail throughout the early afternoon, affecting in particular Office 365 logins.
As we look to welcome teachers, students and parents back online tomorrow, we are confident that the issues faced today are resolved. Our global team will continue to work with our partner Amazon Web Services throughout the night to prepare for Wednesday and thereafter.
We believe the work undertaken will be sufficient for the days to come this term, but we will continue to respond quickly to any unforeseeable issues with frequent and transparent communication with you on progress. We are also already planning to use the Easter Holiday break to strengthen and solidify our infrastructure even further for back to school after Easter.
In the meantime, we would like to share some recommendations that may help smooth your use of Firefly during peak times which include:
- Please encourage students, teachers and parents to consider using our mobile apps, which also work effectively in offline mode.
- Use third-party sharing platforms like ClickView, Planet eStream, YouTube, Vimeo or Microsoft Stream to stream videos rather than uploading them directly to Firefly
Our support team are handing over to our Sydney team shortly and will remain available to help you throughout the night; please bear with us as we deal with the huge volume of questions you’ve all been asking. Finally, for the latest updates on system availability and performance will, please visit and subscribe to: https://fireflylearning.status.io/
This is a tremendous learning curve for schools and education providers alike, including Firefly. We would like to thank you for your patience, understanding and support as we improve the performance and stability of our infrastructure to see us all through the months to come.
Kind regards
Joe & Simon
Co-Founders – Firefly Learning